Re: Superfluous RAID member

[Linux4Bene]

Op Thu, 16 Jul 2015 20:53:54 +0200, schreef Pascal Hambourg:

> Linux4Bene a écrit :
>> 
>> Could it be that grub is confused by the mdadm 0.9 metadata at the end
>> of the disk?
> 
> Maybe. This is typically the kind of problem which can happen with the
> 0.9 superblocks. Why are you using this obsolete format ? You should use
> the newer 1.x format, specifically 1.2 (superblock near the beginning of
> the device) if no adverse requirements. It is the default.
> 
> May I also ask why you created a separate /boot ?

Well, I have tried so many configs to get this dedicated server up and 
running, that I tried a whole bunch of scenarios.
The OVH rescue system only loads raid partitions of the type 0.9.
It didn't look like it wanted to read my type 1.2 raid partitions.
I will try a new install with their installer and see what happens.
Is there a way to convert the 0.9 metadata to 1.2?


>> When I dd'ed, it was only the 40 GB at the start of the disk,
>> not at the end. Any way I can remove this error and not having my LVM
>> data destroyed?
> 
> Maybe, but it depends on how the disks were previously partitionned.
> --metadata= can be added to --zero-superblock to specify which kind of
> superblock you want to erase.

Thanks, I didn't know that.

Regards,
Bene



-- 
To UNSUBSCRIBE, email to debian-user-requ...@lists.debian.org 
with a subject of "unsubscribe". Trouble? Contact listmas...@lists.debian.org
Archive: https://lists.debian.org/mogg56$f9o$1...@ger.gmane.org

Re: Superfluous RAID member

[Linux4Bene]

Op Thu, 16 Jul 2015 10:58:15 +, schreef Linux4Bene:



Could it be that grub is confused by the mdadm 0.9 metadata at the end of 
the disk? When I dd'ed, it was only the 40 GB at the start of the disk, 
not at the end. Any way I can remove this error and not having my LVM 
data destroyed?

Regards,
Bene



-- 
To UNSUBSCRIBE, email to debian-user-requ...@lists.debian.org 
with a subject of "unsubscribe". Trouble? Contact listmas...@lists.debian.org
Archive: https://lists.debian.org/mo86dj$fsf$2...@ger.gmane.org

Superfluous RAID member

[Linux4Bene]

Hi,


I'm installing a server with raid, 2 disks of 2 TB.
There was a previous similar raid config on it. I clear parts of the
disks with dd. Then I create the partitions, make the raids (no errors), 
debootstrap. /boot on /dev/md0, / on /dev/md1 and lvm on /dev/md2.

However, by the time I'm installing grub, I get this error:

error: found two disks with the index 0 for RAID md2.
error: superfluous RAID member (2 found).
error: found two disks with the index 0 for RAID md2.
error: superfluous RAID member (2 found).
error: found two disks with the index 0 for RAID md2.
error: superfluous RAID member (2 found).
...

How is it possible if I create new raid devices, have made a new GPT
table when partitioning with gdisk, and did a dd run for the first 40 
GB's?

The way I create the arrays:

mdadm --create --verbose /dev/md0 --level=1 --metadata=0.90 --raid-
devices=2 /dev/sda2 /dev/sdb2 --assume-clean
mdadm --create --verbose /dev/md1 --level=1 --metadata=0.90 --raid-
devices=2 /dev/sda3 /dev/sdb3 --assume-clean
mdadm --create --verbose /dev/md2 --level=1 --metadata=0.90 --raid-
devices=2 /dev/sda4 /dev/sdb4 --assume-clean

I have found some info, and it said zero the superblocks.
However, if I do that on the md2 raid array, I lose my LVM partitions.

mdadm --zero-superblock /dev/sd[ab]4

$ cat /proc/mdstat

Personalities : [linear] [raid0] [raid1] [raid10] [raid6] [raid5] [raid4] 
[multipath] [faulty] 
md1 : active raid1 sdb3[1] sda3[0]
  10228672 blocks [2/2] [UU]
  
md2 : active raid1 sdb4[1] sda4[0]
  1943027712 blocks [2/2] [UU]
  
md0 : active raid1 sdb2[1] sda2[0]
  252864 blocks [2/2] [UU]
  
unused devices: 

Why am I getting this error? Is there a way to see if this error will
turn up before I proceed to do the rest of the installation?

It's very annoying to encounter this at then end of the install when
the mdadm --create didn't return an error or warning.


Thanks
Bene


-- 
To UNSUBSCRIBE, email to debian-user-requ...@lists.debian.org 
with a subject of "unsubscribe". Trouble? Contact listmas...@lists.debian.org
Archive: https://lists.debian.org/mo82o7$fsf$1...@ger.gmane.org

Re: Free GNU/Linux intro class for teens advice? Purchase box? Squeak/Smalltalk programming

[Linux4Bene]

Op Mon, 06 Jul 2015 20:12:10 -0400, schreef Marc D Ronell:

> I am  working toward  teaching a free  introductory class to  teens on
> GNU/Linux  and the  philosophy of  free  software at  the Newton  Free
> Library in MA this coming September.
> 
> For the class, the participants  will need access to GNU/Linux.  After
> reviewing   some  options,   including   sdf.org,  virtual   machines,
> Chromebooks,  etc.,  I  am  considering just  asking  participants  to
> purchase a dedicated  laptop and installing the OS.  I  may be able to
> direct students to install fests  in the area before the class starts.
> I am  not sure that this is  the best idea, but  it offers significant
> advantages including a potentially working  box as part of the results
> of the course.
> 
> As a test, I purchased  a laptop (Toshiba Satellite C75-B7180) on sale
> for $350  at our local Microcenter  in Cambridge and was  able to load
> GNU/Linux  for my  son.  I  am  thinking of  working some  programming
> assignments in Squeak (Smalltalk), but  maybe C is a better choice for
> an OS class?
> 
> Has  anyone tried  running a  GNU/Linux  intro class  for teens?   Can
> anyone  share their  experiences, thoughts  or  suggestions?  Feedback
> based on actual experience would be most helpful, I think, but I would
> appreciate any insights.
> 
> Thanks for your thoughts,
> 
> Marc

If it's an intro class to Linux, I would expose them to the system first, 
install later. If so, then it could suffice to setup a VPS server and 
make SSH accounts for the students.
You could then even have them make a ssh connection from a Windows box, 
and with the help of Xming run graphical programs from their Win box.

It doesn't cost them any money, and it's easy to setup.
You could focus on teaching instead of installing. When they get the hang 
of the system, you could always go with the laptop route.

As for programming assignments, C has a learning curve.
Python might be better suited, don't know about Smalltalk as I haven't 
used it. You may want to focus on ideas and how to translate those in a 
program, not fight with syntaxes.

Regards,
Bene




-- 
To UNSUBSCRIBE, email to debian-user-requ...@lists.debian.org 
with a subject of "unsubscribe". Trouble? Contact listmas...@lists.debian.org
Archive: https://lists.debian.org/mng859$kmo$4...@ger.gmane.org

Re: / 100% used

[Linux4Bene]

Op Mon, 06 Jul 2015 14:44:41 -0300, schreef Beco:

> Current usage:
> $ du -hc var = 1.1 GB (ext4)
> usr = 8.5 GB (ext4)
> tmp = 200 KB (ext4)
> 
> I'm thinking of:
> var = 10 GB usr = 20 GB tmp = 10 GB
> 
> Or maybe:
> var = 15 GB usr = 20 GB tmp = 5 GB
> 
> And keep all ext4 (to simplify my life, if that is ok, or at least not
> critical).

Using ext4 is not a problem. As for disk sizes, either of your 
suggestions would do, although I would rather spent more disk space on 
var, home or usr then /tmp. Before your tmp was very small so 5 GB should 
do unless you use it to copy very large files, or other users use it all 
the time. Then bigger might be better.


> (**) What configuration tool do you suggest to use for partitioning? Is
> it safe to do it via ssh?

I use gdisk or fdisk. Parted is also well known.

> (***) Should I trust better NetworkManager, or let the server using
> ifupdown? Or change to Wicd?

I edit /etc/network/interfaces myself on my machines and servers.
Works without a problem.

Regards,
Bene


-- 
To UNSUBSCRIBE, email to debian-user-requ...@lists.debian.org 
with a subject of "unsubscribe". Trouble? Contact listmas...@lists.debian.org
Archive: https://lists.debian.org/mng7lb$kmo$3...@ger.gmane.org

Re: Dedicated server doesn't boot, can't seem to figure out why

[Linux4Bene]

Op Tue, 07 Jul 2015 01:17:04 +0200, schreef Pascal Hambourg:

>> I have installed grub,
> 
> In a chroot ?
> Did you properly install the GRUB bootloader on both disks ?
> 
> grub-install /dev/sda grub-install /dev/sdb

The installation is with debootstrap from the rescue prompt.
Then I chroot into the installation, adjust my settings and install grub2.
It asks to what devices I want to install, and I always choose /dev/sda 
and /dev/sdb


> Did you run update-grub in a chroot to create a grub configuration file
> in /boot/grub and did you check it ?

Yes. After the installation of grub which does already do all that,
I have done update-grub, grub-install, and checked the /boot/grub/grub.cfg 
file. I've checked the disk/raid UUID's, compared them and it's all ok.
I made sure the correct UUID is placed in my /etc/fstab also to load boot.
Same method of install works offline in Virtualbox.
 
> Note that GPT is only required on disks larger that 2 TiB. But it's so
> cool even on smaller disks. However, some BIOSes are broken and may not
> boot a standard GPT disk, see
> 

Thanks for the info. The disks are 2 TB. I've tried an installation with 
the OVH installer, and it chose GPT for the disks too, and it booted.
GPT should be ok.


>> only a small EF02 Bios boot partition as I read you need one for some
>> Grub files when using GPT layout.
> 
> Small ? 24 MiB is just huge for this partition. Even 1 MiB is big.

Yeah but one gets a bit paranoia after trying install after install and 
seeing how this install works locally but not on a dedicated server :)


Thanks for the info,

Regards,
Bene


-- 
To UNSUBSCRIBE, email to debian-user-requ...@lists.debian.org 
with a subject of "unsubscribe". Trouble? Contact listmas...@lists.debian.org
Archive: https://lists.debian.org/mng720$kmo$2...@ger.gmane.org

Re: Dedicated server doesn't boot, can't seem to figure out why

[Linux4Bene]

Op Mon, 06 Jul 2015 16:18:04 +0200, schreef claude juif:

> Hi,
> 
> Why don't you use the OVH installer ?

Because the partitioner from the OVH installer doesn't allow me to 
partition the disks how I want. It imposes certain restrictions such as 
putting root in a lv partition, even if you keep the boot partition on 
raid.
It's not very flexible.

Regards


-- 
To UNSUBSCRIBE, email to debian-user-requ...@lists.debian.org 
with a subject of "unsubscribe". Trouble? Contact listmas...@lists.debian.org
Archive: https://lists.debian.org/mng6mb$kmo$1...@ger.gmane.org

Re: Dedicated server doesn't boot, can't seem to figure out why

[Linux4Bene]

Op Mon, 06 Jul 2015 13:50:09 +0100, schreef Darac Marjal:

Hi Darac,


thank you for your response.

> This sounds like a failure in your boot loader. I notice that you say
> you use debootstrap to install the base system, rather than using the
> installer.

Indeed. It's a dedicated server from OVH that I don't have physical 
access to. I don't have the possibility to use the Debian installer.
It would have made it easier, that's for sure.


> Have you remembered to install a boot loader? debootstrap
> normally expects to be installing into a chroot or similar, so won't do
> all the necessary tasks that are needed for a bare-metal install. If
> you've installed grub (I suspect you want grub-efi-amd64, if you're
> using a 64-bit PC), check that the necessary files were loaded into
> /boot/efi (your partition #1 above). You may also get a "Debian" entry
> in your firmware boot selection menu.

I have installed grub, but not grub-efi-amd64. I use a GPT table (not 
MBR), but I don't have an efi parition, only a small EF02 Bios boot 
partition as I read you need one for some Grub files when using GPT 
layout. I'm not sure this requires me to install the grub-efi-amd64 
version? In my off line test in Virtualbox, I was able to install a 64 
bit Debian Wheezy with the standard grub with the same disk layout and it 
booted without problems.

> If, however, grub IS installed, how far through its boot does it get? Do
> you get a "rescue>" prompt? Do you get the menu up? etc.

I don't have physical access and I don't have a way to view/watch the 
boot process. On one occasion the web interfaces' hardware reboot feature 
failed, requiring a technician to reboot the server.
He said he saw a rescue prompt. But that's all the info I have gotten so 
far.

> If you get a "rescue>" prompt, try the following (cadged from the Gentoo
> Wiki):
>   insmod gzio insmod part_msdos   # Probably don't need 
this, actually
>   insmod part_gpt insmod diskfilter insmod mdraid1x insmod ext2 set   
>   root="mduuid/"
>   linux  /boot/vmlinuz blahblah inirtd /boot/initramfs blahblah
> 
> Tab completion should be available, and SHOULD find the UUID of your
> raid (or produce an error, if not, hopefully)
> If you get as far as the menu, do you get any errors when you try to
> select Debian?

See above, I can't perform these actions. I asked them if there is such a 
console, haven't gotten any answer back.


Thanks


-- 
To UNSUBSCRIBE, email to debian-user-requ...@lists.debian.org 
with a subject of "unsubscribe". Trouble? Contact listmas...@lists.debian.org
Archive: https://lists.debian.org/mne1ea$mfj$3...@ger.gmane.org

Re: / 100% used

[Linux4Bene]

Op Sun, 05 Jul 2015 17:32:56 -0300, schreef Beco:


> # apt-get remove wpasupplicant
> 
> I could break the "while" loop deleting syslog. It stoped spamming.
> Looks like all messages was linked to it.
> 
> I don't know very much wpasupplicant. Ill this software be needed in the
> near future?

Only if you plan on using wireless on your server (which seems weird) so 
probably not.

> Im afraid the system is working only now, and will be unreachable next
> reboot.
Then plan a scheduled maintenance where the users know the system won't 
be available for a while. Make sure you have backups of the system.
Notify the users, shutdown the system, disable wireless in the BIOS,
then reboot and fix whatever is on your path.
You might want to do this on premise and have a system at hand connected 
to the net to search for info if you do encounter a problem.

Regards


-- 
To UNSUBSCRIBE, email to debian-user-requ...@lists.debian.org 
with a subject of "unsubscribe". Trouble? Contact listmas...@lists.debian.org
Archive: https://lists.debian.org/mne0qh$mfj$2...@ger.gmane.org

Dedicated server doesn't boot, can't seem to figure out why

[Linux4Bene]

Hi,

I've done several attempts to install Wheezy (can't use Jessie as I want 
to eventually copy a working config over from another server) on an OVH 
dedicated server. It contains 2 x 2TB disks.
I boot into the rescue mode, partition the disks, set-up raid and lvm, 
and then use debootstrap to install the base system. The system 
afterwards fails to boot. The partitioning scheme I made with gdisk (GPT):

Number  Start (sector)End (sector)  Size Code  Name
   1  2048   51200   24.0 MiBEF02  BIOS boot partition
   2 53248  512000   224.0 MiB   FD00  Linux RAID
   3514048  3907029134   1.8 TiB FD00  Linux RAID
   
Partition 2 is for /boot on /dev/md0 using /dev/sda2 and /dev/sdb2. 
Filesystem is ext2. Partition 3 is for all the rest, using LVM on /dev/md1 
(/dev/sda3 and /dev/sdb3).

The raid devices work, lvm works, everything seems fine, except when 
rebooting after the complete install. I can then boot into rescue mode 
but I have no clues as to what went wrong (probably grub?) nor can I 
consult a log file.
I don't seem to have a console to view the boot process. I can only use 
ssh to login to my server or rescue mode.

If I try this exact same setup method locally in Virtual box (same 
install method with debootstrap), it boots without problems. I moved /
boot to it's own raid (not on lvm) because I first thought that was the 
problem. I've tried specifying metadata=0.90 for the raid, and I have 
tried to change the grub settings: GRUB_DISABLE_LINUX_UUID=true
Nothing works and I have no clue why.

I know it's a long shot, but any ideas as to what might be wrong?
Or is there another way I could install with using raid for this server?

Thanks.


-- 
To UNSUBSCRIBE, email to debian-user-requ...@lists.debian.org 
with a subject of "unsubscribe". Trouble? Contact listmas...@lists.debian.org
Archive: https://lists.debian.org/mndpkv$mfj$1...@ger.gmane.org

Move root to lvm after initial install (VPS)

[Linux4Bene]

Hi,


I'm currently moving my VPS to another provider. My current VPS is 
limited to 1024 MB ram, and 50 GB of space. First I was leaning towards 
just using a tar of the old system to install the new VPS. I actually 
want this new system to be ready for high availability with drdb in the 
future or another mechanism.
The machine I got is a 32 GB dedicated server Intel Xeon W3520 with 2x2TB 
sata disks. I want to figure out HA before I even think of offer some 
kind of service to customers. As I'm a start up I don't have the 
resources to spend a lot of money on spare servers but I also don't want 
to deal with crashing sites or servers when I'm not behind a desk.

Currently my VPS server has DNS, email and webserver. In the future, I 
will use separate servers for this, but for now my plan is to setup this 
2nd server first to move the services from my old vps to this one, and
use the old VPS for backup mx, and slave dns. For my website, I haven't 
yet figured out a good way for automatic fail over (pretty new to this) 
besides using a drdb partition and put the site, and the data (Postgres 
db) on there.

In the web interface of the VPS provider (soyoustart) I was able to 
specify raid partitions, for boot and swap. Choices for the partitions 
where primary, logical and lv. However, when I tried to put root on lv, 
it refused. The web interface isn't very handy and it would be more
convenient if I could use the debian installer.
Instead of having a raid partition with lvm for /, /usr, /var, /tmp, I 
end up with this:

sda   8:001,8T  0 disk
├─sda18:10 1004,5K  0 part
├─sda28:20511M  0 part
│ └─md2   9:20511M  0 raid1 /boot
├─sda38:30   17,6G  0 part  [SWAP]
├─sda48:40 30G  0 part
│ └─md4   9:40 30G  0 raid1 /
└─sda58:501,8T  0 part
  └─md5   9:501,8T  0 raid1
├─vg-usr (dm-0) 254:00 20G  0 lvm   /usr
├─vg-var (dm-1) 254:10 30G  0 lvm   /var
└─vg-tmp (dm-2) 254:20  5G  0 lvm   /tmp
   
Below is what I have on my local test system, and what seems like 
something useful. I would use this layout for my new VPS (increased sizes 
of course) and leave the rest of the space for a drdb partition or 
whatever solution I end up with for failover.

sda8:0010G  0 disk 
├─sda1 8:10   487M  0 part 
│ └─md09:00 486,7M  0 raid1 /boot
├─sda2 8:20   954M  0 part 
│ └─md19:10 953,4M  0 raid1 [SWAP]
└─sda3 8:30   8,6G  0 part 
  └─md29:20   8,6G  0 raid1
├─vg0-vg_root (dm-0) 253:00   2,8G  0 lvm   /
├─vg0-lv_usr (dm-1)  253:10   2,8G  0 lvm   /usr
├─vg0-lv_var (dm-2)  253:20   2,6G  0 lvm   /var
└─vg0-lv_tmp (dm-3)  253:30   396M  0 lvm   /tmp

Some questions:   
- Is there a way to still get / on lvm on raid even if the web installer 
doesn't seem to like it?

- Any easy way to get fail over for my site? I do have a small database 
coupled to it. rsync seems possible but I doesn't seem safe to copy a 
running db with it. And it still leaves me needing a mechanism to detect 
when my site/server is down.

- Seems like I'll need 2 load balancers and 2 servers to really be safe 
in the future. I think I would need at least 1 extra IP for the load 
balancers, and use that IP adres in my DNS settings for my site/email 
server. This way, the load balancers (HA Proxy, ...) get the requests, 
and can forward the request to the right server. Is this correct?

- I've thought about kvm or linux containers to seperate the services on 
the VPS. Not sure if these could be handy in my case. If I use containers 
of some sort, I also need to get the data replicated to another server.
My old VPS is rather slow, and only has 50GB of space so my options are 
limited. If I offer services, I will get another VPS similar to this new 
VPS.

- If I would implement HA in the future, is a backup mx still useful?
I think it is. If so, that would mean having another server with just the 
backup mx?

Thanks for any info,
Regards


-- 
To UNSUBSCRIBE, email to debian-user-requ...@lists.debian.org 
with a subject of "unsubscribe". Trouble? Contact listmas...@lists.debian.org
Archive: https://lists.debian.org/mmaujv$upn$1...@ger.gmane.org

Re: Moving server to new server with tar

[Linux4Bene]

Op Mon, 08 Jun 2015 16:06:41 -0600, schreef Bob Proulx:

> Linux4Bene wrote:

> Since /dev is dynamic anything done to it will evaporate after a reboot.
>  After a reboot it will all be as if nothing had been overwritten there.
>  If you get to the point of a reboot then there is nothing lingering
> afterward.  You would be in the clear.  I would still exclude it of
> course.  The possibility of archiving /dev/mem for example makes me
> nervous. :-)

Hehe indeed. I will change the tar command to exclude it completely.


>   http://marc.merlins.org/perso/linux/post_2014-01-06_My-Live-Upgrading-
>Many-Thousands-of-Servers-ProdNG-talk-at-Linux_conf_au-2014.html
> 
> Unfortunately the original paper is now 403 forbidden.  I think that is
> likely a mistake somewhere.  But the Internet Archive Wayback Machine
> has a copy if you want to browse it.
> 
>   https://web.archive.org/web/*/http://marc.merlins.org/linux/talks 
>ProdNG-LCA2014/Paper/ProdNG.pdf

Thanks, the last links you posted worked. I'm very interested in reading 
about it.


> As I recall one spot I don't think was as complete was the kernel both
> running and otherwise.  I think debtakeover required that the new system
> run on the foreign system's kernel.  Which is not always possible due to
> system version differences.  I think that is more of a problem now than
> it was then.  So for example replacing a RHEL 6 system with Jessie would
> fail because jessie binaries require a newer kernel than the default
> RHEL 6 kernel.  But probably upgrading the kernel first with a native
> backport and then doing the debtakeover process would get past that
> problem.

It still seems like software that can be useful today or are there other 
preferred (manual) ways of doing such a conversion?

>> When I rebooted the system, it failed because the UUID was still the
>> UUID of the main disk of the old system.
> 
> Ah...  Probably should 'grep -r $UUID /etc' for every mention of it.

Another useful comment, thanks.

> Are you using lvm or raid?  If either of those then would probably want
> to *avoid* overwriting the /etc/lvm or /etc/mdadm directories. Both of
> those configs keep UUIDs in them.

No, it's a fairly simple VPS, no raid or LVM.


>> Bob, thanks for your thorough explanation & insight.
> 
> It is an interesting problem.  I am deep in the middle of installation
> and setup all of the time.
> 
> Bob

Sounds very challenging and interesting at the same time :)


Benedict


-- 
To UNSUBSCRIBE, email to debian-user-requ...@lists.debian.org 
with a subject of "unsubscribe". Trouble? Contact listmas...@lists.debian.org
Archive: https://lists.debian.org/ml62l8$d52$1...@ger.gmane.org

Re: Moving server to new server with tar

[Linux4Bene]

Op Sat, 06 Jun 2015 13:59:16 -0600, schreef Bob Proulx:
 
> This is one of those hard topics.  It seems easy enough.  But the
> reality is that there are many subtle problems.  It comes up for
> discussion every so often over the years.  I don't think there has ever
> been a completely satisfactory canonical one true answer that solves the
> problem in one collection.  There just isn't any perfect methods. 
> Instead all that we can suggest is to understand everything and deal
> with it ad-hoc.  This is exactly what you are doing.  With what you have
> written I can tell that you are very much aware of most of the traps and
> pitfalls and are dealing with them as best as you can.  Good job!
> 
> It would be good if there were a better way to deal with this.  There
> are many different strategies.  Some people favor one strategy.  Other
> people favor other strategies.  You and I have already diverged from
> favored strategy.  Personally I prefer to build a pristine system and
> then install the new services upon it.  That allows me to be refreshed
> in everything.  (On the other hand I always upgrade servers in place. I
> carry the upgrade history along with me.  An irony of opposites for me I
> know.)  I would use this as an opportunity to clean and clean and clean.
>  But it is okay if you tell me you want to have the identical server, as
> identical as possible, moved without doing that in this step.  That is
> fine too.

Bob,

thanks for your reply and the time invested. Much appreciated.
It does indeed seem tricky unless you go the full monty and replace the 
whole installation except for the special dirs like dev as you noted.
In my test, I didn't get any strange results in the end but I have 
learned not to always trust what I see in IT. In the back of my mind, I 
always suspect a problem popping up when the server is in production.

 
>> Tar command from the backup script on the old server:
>> EXCLUDE="--exclude=proc --exclude=sys --exclude=dev/pts
>> --exclude=backups"
>> tar -czpf /backups/full.tar.gz --directory=/ $EXCLUDE / 2>&1
> 
> Since /dev is dynamic I exclude /dev from the backup too.  Your new
> installation will already have a static copy of the minimum dev under
> the udev mount point.

Indeed, it should be excluded as well. Good point.

 
> One sideways strategy that you might consider for risk management is to
> untar a copy of your old system into what will become a chroot area on
> the new system.  That will give you a reference on the new system. You
> can run services from there.  But mainly it would give you a way to do
> an A-B comparison between what you had before and what you are creating
> new.  I do that often.  If something shows up being different then can
> go investigate the way things were and find lost and forgotten tweaks
> and revive them.

That's a really good idea. Untar in a separate directory and manage the 
installation of the services from there or use it as a base for 
comparison. I like it.

 
> Right.  They intentionally confict with each other and push each other
> out.  It will sound obvious but postfix can't come up if it isn't
> installed.  :-)
> 
> But if you were overwriting *everything* on the system from the backup
> on one to the new system then after having done so then postfix would
> have been installed.  Right?  The binaries in /usr/sbin/postfix would
> have been copied into place and the package manager would think it had
> installed it in /var/lib/dpkg too.  The biggest issue being any daemon
> that changed uids and was running would need to have been stopped before
> this and restarted after this.  Right?  This is one of the issues that
> makes doing it this way tricky.  Not impossible.  Just tricky.

The reason why postfix didn't want to start, even after untarring the 
whole system was because exim was still running. After stopping exim, I 
could start postfix without a hitch.

 
>> Might be better to start with:
>> Old server: dpkg --get-selections > packages
>> New server: dpkg --set-selections < packages
> 
> Yes.  That is what that was designed for along with some other things
> such as the dselect-upgrade and so forth.  Those will be suggested and
> with identical versions (Stable, OldStable) they should be able to
> replicate the same packages installed on each machine.
> 
> There are some issues with doing it this way.  You should read about
> 'apt-mark' and the database flag that indicates whether the package has
> been installed explicitly or automatically.  Automatically installed
> packages without anything depending upon them are candidates for
> 'apt-get autoremove' to remove them.  Explicitly installed packages are
> not.  You can dump the previous values from the old server with:
> 
>   apt-mark showauto 
>   apt-mark showmanual
> 
> And then use the lists to set the same values in the new server.
> Here is one way.  I will leave you to fiddle with this further.
> 
>   apt-mark auto $(cat list-of-auto-packages)
> 
> You can

Moving server to new server with tar

[Linux4Bene]

Hi,


I am in the process of moving my server to another VPS.
The goal is to keep the old VPS around and convert it to backup MX & DNS 
amongst other things. I will purchase the new VPS from another company so 
I can't just copy the vm file/container.

As a start, I would do a full tar archive of the old server and start 
from there. A test on a local VM worked, with some adjustments. Both use 
Debian 7.8. The services on the old server that need to me moved:
- Mail: Postfix, Dovecot, Spamassassin, Clamav, Postgresql, ...
- Web: nginx, supervisord, python, php5-fpm, Postgresql, ...
- DNS: PowerDNS

Tar command from the backup script on the old server:
EXCLUDE="--exclude=proc --exclude=sys --exclude=dev/pts --exclude=backups"
tar -czpf /backups/full.tar.gz --directory=/ $EXCLUDE / 2>&1

I know I can migrate by first installing all packages, and then copying 
the config and data from one server to the other. But then you need to 
pick all data to be moved. It takes longer and it's more prone to error 
(forgetting something). I want this server to be exactly the same as the 
first one.

What I've found so far in my test:
- It's a good thing to first install all the same packages on the new 
machine first. I didn't do that in my first test and Postfix wouldn't 
come up because of Exim that was installed on the base version of the new 
OS. Simple to solve but this wouldn't have happened if I had installed 
Postfix first as Exim would have been purged.
Might be better to start with:
Old server: dpkg --get-selections > packages
New server: dpkg --set-selections < packages

- Extract the tar archive from the root on the new server
- Adjust /etc/hostname, /etc/hosts, /etc/network/interfaces
- Adjust PowerDNS settings on new server. If the new server is up I will 
need to change the PowerDNS settings on the old server as well and set up 
DNS synchronization. DNS entries at the domain registrars & reverse DNS 
will need to be changed.
- Check configs of the services above for the old ip or hostname
- Run update-grub as the id of the disk has changed.
- Reboot

This worked well, but I wonder if there are good reasons to not do move
the server like this?

Thanks for any info or insight

Regards,
Benedict


-- 
To UNSUBSCRIBE, email to debian-user-requ...@lists.debian.org 
with a subject of "unsubscribe". Trouble? Contact listmas...@lists.debian.org
Archive: https://lists.debian.org/mkovvi$jc2$1...@ger.gmane.org

Re: iptables rules disappear

[Linux4Bene]

Op Wed, 25 Mar 2015 11:46:21 +0100, schreef Diogene Laerce:

> Hi,
> 
> I have a strange behavior of iptables lately : all rules are cleaned up
> after a few minutes.
> 
> iptables-persistent is installed and if I reboot just after restoring
> all rules,
> all rules are still loaded. But a few minutes later, they are all
> cleaned up.
> The same happens whenever I load them manually.
> 
> The chains are cleaned up but are still present, they are just empty.
> 
> Does someone have a idea how I can debug this ?
> A "grep -R iptables /var/log" didn't return anything relevant.
> 
> Thank you


Whenever I set up iptables I use /var/log/syslog to debug.
You haven't installed a script that might flush the iptables?

Do you have a intrusion detection system running?
I have running to see what changes on my system.

Regards,
Benedict


-- 
To UNSUBSCRIBE, email to debian-user-requ...@lists.debian.org 
with a subject of "unsubscribe". Trouble? Contact listmas...@lists.debian.org
Archive: https://lists.debian.org/meubsa$74m$1...@ger.gmane.org

Re: Advise on setup of small office locally or via VPS

[Linux4Bene]

Op Tue, 17 Mar 2015 18:50:39 -0700, schreef David Christensen:

> On 03/17/2015 04:22 AM, Linux4Bene wrote:
>> Thanks for any advice, thoughts, links or info and for your patience if
>> you got this far :)
> 
> I run a SOHO LAN with ADSL, 4 static IP's, and a few Internet services.
> 
> 
> I avoid running key Internet-facing services locally -- my WAN bandwidth
> is too precious and the services are too important.  I prefer service
> provider DNS and mail, and VPS WWW.

I thought about using the domain registrars DNS but I wanted to set it up 
as a learning exercise. VPS is really suited for www.
I still have to figure out how to setup a staging area. Do I go with 
another VPS server for that or not? Ideally there would be another 
machine hosting the sites so they are still accessible when the other VPS 
goes down.
I haven't really researched this yet, but it's on my to do list.


> +1 for using a dedicated device/ FOSS distribution for your WAN/LAN
> gateway.  I use IPCop.

I have heard of IPCop, haven't tried it out.

> +1 for using Samba for the LAN file server -- I want interoperability:
> Linux, *BSD, Windows, Mac, and others.

Indeed, and the setup is rather painless :)

> VPN's are appealing, but consider the consequences of a VPN machine
> compromise.  Securing the rest of the VPN against that risk is
> non-trivial, and involves other people's computers and networks.  I
> turned it off.

I thought it made some sense to tie the WAN and LAN part together.
After reading your comment, it indeed seems like over complicating things.
As Dan already suggested, there is merit in KISS.

I guess you access your VPS servers also via SSH only then?
I run no gui on them so it's enough for my needs.


Thanks David for the insight,

Regards,
Benedict


-- 
To UNSUBSCRIBE, email to debian-user-requ...@lists.debian.org 
with a subject of "unsubscribe". Trouble? Contact listmas...@lists.debian.org
Archive: https://lists.debian.org/mebflf$dbt$2...@ger.gmane.org

Re: Advise on setup of small office locally or via VPS

[Linux4Bene]

Op Wed, 18 Mar 2015 03:58:02 +, schreef Dan Purgert:


> I read it as you were /planning/ on using a Debian box for routing and
> firewall (and then switched gears to "what's a good appliance?" midway
> through the writing), which is why I asked.
> 
> Honestly, unless you already have said box ready to go, I would skip it
> and just use an appliance (e.g. the UBNT Edge Router).  Less to go wrong
> / muck up.

I don't have such a box so I would rather use an appliance as you 
suggested.


>> Thanks, looks like a simple and adequate solution.
> 
> Yeah, they're a bit more than "adequate" -- they rival equipment put out
> by other vendors that's several times more expensive (IIRC, "cheap"
> Cisco kit is like 500-1000 USD).

Yes, I really liked the specs.

> Note - I'm in the USA, perhaps your local ISP's equipment isn't as
> rubbish as the ones here.  Best way to figure it out is by finding out
> what they'd supply, and then digging up discussions about it on google.

Indeed, I will look at the router type and see what google comes up with.

> What I meant was that if you're putting a "local" server into a DMZ area
> already (because it's public facing), adding that extra internal server
> seems to be adding complexity for the sake of complexity, and wouldn't
> be offering you any benefits -- this also ties in with your webmail
> solution, if you choose to also have that going.
> 
> Now, if you were a bigger company with two or more sites that happen to
> be somewhat distant from one another, then running a relay would be
> beneficial (as users would all be hitting their "local" mail server,
> instead of /everyone/ needing to hit the server at your HQ site).

That's a valid remark. I will opt to leave the mailserver on the VPS for 
the time being.

> You've already got a frontend for them (hint - "roundcube")

Yes, I just need to find a good plugin allowing for the users to change 
their password.


> Probably not.  I mean, yeah some of the syntax for the config files may
> have changed, but LDAP is still LDAP ... so the core principles of the
> setups will be the same.

I dug up my notes and I have found some ldif files and procedures.
I'm good to go.

> emacs :)

Hehe, I have tried it once. I should take the time to give it a more 
thorough try.

> Git works well with source code, I'm not really sure how well it works
> outside of that (e.g. ODT files).  I imagine that it would provide
> "some" of the functionality you're looking for, but possibly not all of
> it.
> 
> For simple text files, I've taken a liking to rcs.  One of the guys here
> (or on one of the other newsgroups I haunt) had a decent basic wrapper
> for it too.

I don't know rcs. I will have a look at it.

> Well, not so sure about the extra firewall in the mix there - I mean,
> yeah you'll have one on site likely as part of your router appliance ...
> but that's pretty much a given these days anyway.
> 
> Or are you planning on throwing a firewall somewhere else, such as
> between the LAN and the file server (and if so - why?)

I would hook up the firewall after the ISP router, before the LAN.
The routers of ISP's here only have very basic firewall capabilities.
I rather use my own device to protect the LAN.
And it gives me a chance to learn the UBNT Edge router.

> They'll definitely make it to your ISP.  Whether or not your ISP will
> relay them as "yourdomain.com" or
> "our-ip-address-block.somewhere.ISP.com"
> is something you'll have to check with them though ...
> 
> Really about the only guaranteed way of getting that would be to own an
> actual block of IPs (i.e. bought directly from one of the number
> registrars ... ARIN or RIPE or one of their delegated subsidiaries). 
> But in doing so, you're talking about buying something like a /20 (or
> whatever their currently "smallest" allocation is).

A big block is going to be overkill so I'll have to get by with whatever 
my ISP offers me. If I have a couple of IP's, it's enough for the public 
services I have.

Regards,
Benedict


-- 
To UNSUBSCRIBE, email to debian-user-requ...@lists.debian.org 
with a subject of "unsubscribe". Trouble? Contact listmas...@lists.debian.org
Archive: https://lists.debian.org/mebf48$dbt$1...@ger.gmane.org

Re: Advise on setup of small office locally or via VPS

[Linux4Bene]

Op Tue, 17 Mar 2015 13:38:26 +, schreef Dan Purgert:



> Didn't you just say that you were using a Debian box as your firewall/
> router?

Not yet. I'm still employed but have everything up and running in a VPS,
and I have all the legal stuff in order like VAT and so on.
Legally this means it's seen as a secondary activity.
>From the moment I quit, it becomes my main occupation.
That's how it works over here.

Currently I have my own VPS running but no business internet line yet ror 
a Debian Firewall but that's the plan. Just thinking ahead on how I will
get up and running as fast as possible :)

> Personally I have used Ubiquiti Edge Routers (ubnt.com), and they're
> really nice - based on Vyatta 6.3, rival bigger names in terms of
> routing performance, and are cheap ($100 for the 3-port model "ER Lite",
> and under $500 for the 8-port "ER-8".  There's also a "PRO" variant of
> the 8-
> port that includes 2 SFP ports that're shared with 2 of the copper
> ports,
> and a 5-port model with PoE, but this is really only the ER Lite with a
> switch in the same case, so it's 2x routing ports + 3x switch ports, and
> might not fit in your situation).
> 
> Here's the Datasheet for their routers -->
> http://dl.ubnt.com/datasheets/edgemax/EdgeRouter_Lite_DS.pdf

Thanks, looks like a simple and adequate solution.

> It's not "difficult" to get redundancy, though depending on the levels
> of redundancy you're after, it can get a bit complex.
> 
> Easiest route is a cold spare -- buy a second of whatever router, config
> it exactly the same way, and then shut it down for use if / when the
> first one dies.
> 
> Though you could always scale to multiple WAN connections spread across
> multiple routers, with OSPF / iBGP being used to manage the routes...
> but this is probably a bit much for a small business.
> 

I should have been more clear about the use case. The cold spare in my
case is enough. If a lot of other people would use services, that's
somethings else but I don't see that happening in the near future.


> Depends on how their router is configured, but this sounds about right.
> That said, in 99.5% of cases that I've seen the ISP-provided routers are
> absolute rubbish, and should be relegated to bridge-only mode so that
> you can use a better (i.e. more configurable) device to handle the
> tasks.

I didn't know that. Thank you for the information.

> If the email server is public already (in the DMZ zone), you'll probably
> have an easier (and still secure) time if you just have the clients
> using STARTTLS to access THAT server.  Not that you couldn't set up a
> gateway /
> relay, but there is much to be said about the KISS principle.

The mail service is public on the VPS. There isn't a DMZ zone on that 
server. As you suggest, both postfix and Dovecot are accessible via 
STARTTLS/SSL. If I read your comment correctly, you would leave the
mail server config as it is, and put it in a DMZ and that's it?
This would leave the mails also in the DMZ but as you said, accessing mail
can only be done over a secure connection (SSL).
I have SSL certificates setup for this (for my website, and Dovecot).

>> - I have Roundcube (webmail) installed as well. I think I could handle
>> this by forwarding the requests from firewall to the internal mail
>> server.
>> Not sure if this is the safest way to do this.
>> One can of course argue about web mail in the first place.
> 
> Again, might be easiest (best) to keep the entire mail service in the
> DMZ, including webmail.

OK I would really like to go KISS :)
Basically, if I end up with a local situation I would move the services 
to a local server in a DMZ zone. Otherwise, I could just keep the VPS
to serve as our mail server.

>> - Central user and document management.
>> I would like to have a space on the file server where people could
>> store their own and shared documents. I think I would need NFS for this
>> (haven't used this before). The docs might need to be accessible from
>> Windows as well, although I really would like to only use Debian
>> machines for my own people. Otherwise, this would mean using Samba.
> 
> If you need / want access to the file server from windows hosts, I'm
> pretty sure samba is your only solution.

That's what I thought.
 
>> My mail users are in a Postgresql database. I would like to keep it
>> that way if I would ever provide mail to customers.
> 
> Sure. If you're selling email services, then you might need a dedicated
> DB box, but that's not exactly 'difficult'.

Indeed. There is some really great info regarding Postfix and keeping
all the necessary info in a Postgresql db. If I would ever go with
offering this as a service to users, I would use Django to build a web 
interface but that's a whole different topic.

In my current mail setup, I would need to provide a way for users to 
change their password. Maybe Roundcube has such a plugin.


>> I can see LDAP being useful to have central authentication.
>> It can be a chal

Advise on setup of small office locally or via VPS

[Linux4Bene]

Hi,


sorry in advance for the lengthy post.
I have some questions on organizing and designing a small office 
environment. Clients and server parts Debian. I have always introduced 
Debian in every job I had in the last 14 years, and it would be great to 
finally use them as the default OS on devices of my own business :)

I currently have one VPS with a few services: hosting my own websites and 
DNS (authoritative for my domains), mail (Postfix,Dovecot). As I'm 
planning to start my own business, I would like to inform myself on the 
available choices.

I would probably get a business vdsl line, which would give me 8 public 
IP's. I have experience with most of the techniques described below, 
although it has been a while since I used some of those components/
software. I do manage some Debian servers, and have done so for the last 
14 years.

At the start, I would only employ 1 or 2 people. I'm trying to keep it 
small so I wouldn't want to go over 10 people.
Server part Debian, office parts also Debian as much as possible but we 
will also have MS machines as we need this to support our clients. Not 
sure if we would need to access any info on the Debian machines or 
servers. I have no preference to local infrastructure as opposed to cloud.

That's why I started out with a VPS to host my sites, mail and DNS.
Because of the DNS redundancy requirements, I use a free service that
replicates my DNS. Ideally, I would be able to provide this redundancy 
with my own machines, VPS'es or local.
I would like your advise on the way I would set this up locally or with 
VPS'es.

Local setup
===
I would connect a Debian box with 3 nics to the ISP router to serve as
firewall. 1 nic for WAN, 1 for LAN, 1 for DMZ. I have always used 
iptables to do this. The wan nic would have 1 public IP, LAN 
192.168.1.0/24,
DMZ 172.16.1.0/24.

DMZ would have 2 machines: 1 with web and DNS 1, another with DNS 2 and 
SMTP gateway. I would keep the free DNS for added redundancy. On the LAN 
part, I would put a file server, local DNS and some internal web apps.

This raises some questions:
- What device could I use for the firewall. I don't want to use an old
computer as I have some public services and need a reliable service.
I'm open to using an appliance as well. Any links or info is welcome.
Any easy way to having this devices redundant?

- I would only allow some traffic (mail for instance) from the DMZ to the
private LAN. LAN could access the DMZ. Any downside to this security wise?

- If I have multiple public IP's, I would assign each public machine a 
public IP. I assume it's the ISP's job to redirect the IP's in my range 
to their router in my office. I could then map the public IP's to a 
private IP by prerouting all allowed traffic on the public IP to the 
private IP address of the machine in the DMZ.

- My mail service (only used for my own purposes right now) consists of
Postfix, Clamav, Pyzor, Razor, Spamassassin, with authentication provided 
by Dovecot. Domains, users and aliases are stored in a Postgresql 
database. Security wise it would be better to place this set up in the 
LAN part, and put a SMTP gateway in the DMZ to receive mail, and have the 
gateway forward the mail to the setup I just described.
The SMTP gateway should have the same parts (Clamav, Spamassassin, ...) 
but just not store the mail locally. Any thoughts on this kind of setup?

- I have Roundcube (webmail) installed as well. I think I could handle 
this by forwarding the requests from firewall to the internal mail server.
Not sure if this is the safest way to do this.
One can of course argue about web mail in the first place.

- Central user and document management.
I would like to have a space on the file server where people could store 
their own and shared documents. I think I would need NFS for this 
(haven't used this before). The docs might need to be accessible from 
Windows as well, although I really would like to only use Debian machines 
for my own people. Otherwise, this would mean using Samba.
My mail users are in a Postgresql database. I would like to keep it that
way if I would ever provide mail to customers. 
I can see LDAP being useful to have central authentication.
It can be a challenge to setup though. Are there other ways of having a 
simple central authentication?

I have thought about using a document management system from the start.
But I have only experience with commercial ones and that might be overkill
from the start. Besides, they are Windows based.

VPS
===
The other way I could go is by using multiple VPS servers (or renting 
dedicated servers). I could connect them with OpenVPN. I have no 
experience with that.
But this would also mean I would have my file server online.
Then I definitely would need to setup a permanent connection from the 
office firewall to the online servers. 

Might make it a bit harder to fully manage reverse dns. As for my current 
VPS, I had to ask my VPS supplier to insert a reverse 

dpkg error installing ncurses deb made with checkinstall

[Linux4Bene]

Hi,

I have previously compiled ncurses 5.9 without a problem.
The config command:
./configure --with-shared --enable-termcap --prefix=/usr/local

make install also works on my Debian Wheezy.
I want to use this same package on my server, so I decided to build a deb
using checkinstall. Building is done with this command:

sudo checkinstall -D --pkgname=ictforce-ncurses-5.9 make install

However, when I try to install the package I get an error on a man page:

Uitpakken van myncurses-5.9 (uit .../myncurses-5.9_5.9-1_amd64.deb) ...
dpkg: error processing /home/user/src/ncurses-5.9/
myncurses-5.9_5.9-1_amd64.deb (--install):
 unable to open '/usr/local/share/man/man3/menu_new.3menu.gz.dpkg-new': 
Bestand of map bestaat niet
 Processing triggers for man-db ...
 Fouten gevonden tijdens behandelen van:
  /home/user/src/ncurses-5.9/myncurses-5.9_5.9-1_amd64.deb

I have tried copying another man page to the location dpkg complains 
about, but that doesn't work. Neither does creating an file in the ncurses 
man directory prior to running the checkinstall command.
To me, I couldn't care less about the referenced man page, I don't need 
it. I can see 2 solutions that could work for me:

1. force install the package as I don't care about the man page and the 
man page is not going to hinder the way ncurses runs.

2. get the missing man page in the package so dpkg doesn't complain when 
installing.

Any ideas?

Thanks,
Bene 


-- 
To UNSUBSCRIBE, email to debian-user-requ...@lists.debian.org 
with a subject of "unsubscribe". Trouble? Contact listmas...@lists.debian.org
Archive: https://lists.debian.org/m9gfjo$6mu$1...@ger.gmane.org