Petr Bena's Bl0g

How to install mdadm to XenServer 7

June 6, 2017
computers, Linux, ...
2 comments

Based on https://discussions.citrix.com/topic/378478-xenserver-7-raid1-mdadm-after-install-running-system/

# 1. Install Xenserver 7 with normal single disk configuration, don't create SR storage
# 2. copy partition talbe from sda to sdb
 
# !!! important don't write the order wrongly, from sda to sdb is like the following
sgdisk /dev/sda -R /dev/sdb


# Important! The partition layout may differ with XenServer version, basically there are 2 partitions with same size for OS and backup
# and at least 1 for GRUB and 1 for swap

parted /dev/sdb
# print
# quit

# You should see a list of partitions, one of them will be flagged as legacy_boot, grub, let's call it BOOT

## flag them as raid disks for sdb partitions
sgdisk --typecode=1:fd00 /dev/sdb # OS
sgdisk --typecode=2:fd00 /dev/sdb # Backup OS
sgdisk --typecode=3:fd00 /dev/sdb # ??
sgdisk --typecode=4:ef02 /dev/sdb # BOOT
sgdisk --typecode=5:fd00 /dev/sdb # LOGS
sgdisk --typecode=6:fd00 /dev/sdb # SWAP

## note that BOOT partition is not the same like the others. Because in the new disk configuration they changed the boot partition.

# 5. create the software raid partitions

mdadm --create /dev/md0 --level=1 --raid-devices=2 --metadata=0.90 /dev/sdb1 missing
mdadm --create /dev/md1 --level=1 --raid-devices=2 --metadata=0.90 /dev/sdb2 missing
mdadm --create /dev/md2 --level=1 --raid-devices=2 --metadata=0.90 /dev/sdb3 missing
mdadm --create /dev/md3 --level=1 --raid-devices=2 --metadata=0.90 /dev/sdb4 missing
mdadm --create /dev/md4 --level=1 --raid-devices=2 --metadata=0.90 /dev/sdb5 missing
mdadm --create /dev/md5 --level=1 --raid-devices=2 --metadata=0.90 /dev/sdb6 missing
mkswap /dev/md5

# 6. copy the contents of / and /var/log directories to the new partitions

mkfs.ext3 /dev/md0
mkfs.ext3 /dev/md4

# 7. mount newly created/formatted partitions
mount /dev/md0 /mnt
mkdir -p /mnt/var/log
mount /dev/md4 /mnt/var/log

# 8. copy contents to the newly mounted directory

cp -xR --preserve=all / /mnt

# 9. create a mdadm file for boot process (!!!if you forget the file the MD devices will have different names)

### the head of the file should include these lines
echo "MAILADDR root" > /mnt/etc/mdadm.conf
echo "auto +imsm +1.x -all" >> /mnt/etc/mdadm.conf
echo "DEVICE /dev/sd*[a-z][1-9]" >> /mnt/etc/mdadm.conf
mdadm --detail --scan >> /mnt/etc/mdadm.conf

# 10. copy the contents to the root folder
cp /mnt/etc/mdadm.conf /etc

# 11. configure mount points
sed -i 's/LABEL=root-[a-zA-Z\-]*/\/dev\/md0/' /mnt/etc/fstab
sed -i 's/LABEL=swap-[a-zA-Z\-]*/\/dev\/md5/' /mnt/etc/fstab
sed -i 's/LABEL=logs-[a-zA-Z\-]*/\/dev\/md4/' /mnt/etc/fstab
sed -i '/md5/ a\/dev/md5          swap      swap   defaults   0  0 ' /mnt/etc/fstab
cp /mnt/etc/fstab /etc

# 12. change the label name for /dev/sdb1 partition
e2label /dev/sda1 |xargs -t e2label /dev/sdb1

# 13. bind mount dev sys proc to the mnt folder
mount --bind /dev /mnt/dev
mount --bind /sys /mnt/sys
mount --bind /proc /mnt/proc
chroot /mnt  /bin/bash

# 14. install grub on /dev/sdb
grub-install /dev/sdb

# 15. backup initrd
cp /boot/initrd-$(uname -r).img /boot/initrd-$(uname -r).img.bck

# 16. create new initrd for raid
dracut --mdadmconf --fstab --add="mdraid" --filesystems "ext3 tmpfs devpts sysfs proc" --add-drivers="raid1 raid456 mdraid1x mdraid09" --force /boot/initrd-$(uname -r).img $(uname -r) -M

###never change the boot configuration via grub-mkconfig.. it will kill xenserver.. change the GRUB configuation, by hand inside the files

# 17. change grub configuration
sed -i 's/quiet/rd.auto rd.auto=1 rhgb quiet/' /boot/grub/grub.cfg
sed -i 's/LABEL=root-[a-zA-Z\-]*/\/dev\/md0/' /boot/grub/grub.cfg
sed -i '/search/ i\   insmod gzio' /boot/grub/grub.cfg
sed -i '/search/ i\   insmod part_msdos' /boot/grub/grub.cfg
sed -i '/search/ i\   insmod diskfilter mdraid09' /boot/grub/grub.cfg
sed -i '/search/ c\   set root=(hd0,gpt1)' /boot/grub/grub.cfg

# 18. exit from chroot
exit

# 19. change the same things in sda1 partition so after reboot you don't need to boot from second disk
cp /mnt/boot/initrd-3.10.0+10.img /boot/

sed -i 's/quiet/rd.auto rd.auto=1 rhgb quiet/' /boot/grub/grub.cfg
sed -i 's/LABEL=root-[a-zA-Z\-]*/\/dev\/md0/' /boot/grub/grub.cfg
sed -i '/search/ i\   insmod gzio' /boot/grub/grub.cfg
sed -i '/search/ i\   insmod part_msdos' /boot/grub/grub.cfg
sed -i '/search/ i\   insmod diskfilter mdraid09' /boot/grub/grub.cfg
sed -i '/search/ c\   set root=(hd0,gpt1)' /boot/grub/grub.cfg

# 20. reboot

# !!!!!!reboot the server the server will boot from software raid..!!!!!
# 21. After the reboot add the /dev/sda to the new MD disks.

sgdisk /dev/sdb -R /dev/sda

mdadm -a /dev/md0 /dev/sda1
mdadm -a /dev/md1 /dev/sda2
mdadm -a /dev/md2 /dev/sda3
mdadm -a /dev/md3 /dev/sda4
mdadm -a /dev/md4 /dev/sda5

# This will take a while for resync of all disks

grub-install /dev/sda

# Create SR

xe sr-create content-type=user device-config:device=/dev/md2 host-uuid=<host-uuid> name-label=”SRRaid1-Local” shared=false type=lvm

100

101

102

103

104

105

106

107

108

109

110

111

112

113

114

115

116

117

118

119

120

121

122

123

124

125

126

127

128

129

# 1. Install Xenserver 7 with normal single disk configuration, don't create SR storage

# 2. copy partition talbe from sda to sdb

# !!! important don't write the order wrongly, from sda to sdb is like the following

sgdisk /dev/sda -R /dev/sdb

# Important! The partition layout may differ with XenServer version, basically there are 2 partitions with same size for OS and backup

# and at least 1 for GRUB and 1 for swap

parted /dev/sdb

# print

# quit

# You should see a list of partitions, one of them will be flagged as legacy_boot, grub, let's call it BOOT

## flag them as raid disks for sdb partitions

sgdisk --typecode=1:fd00 /dev/sdb # OS

sgdisk --typecode=2:fd00 /dev/sdb # Backup OS

sgdisk --typecode=3:fd00 /dev/sdb # ??

sgdisk --typecode=4:ef02 /dev/sdb # BOOT

sgdisk --typecode=5:fd00 /dev/sdb # LOGS

sgdisk --typecode=6:fd00 /dev/sdb # SWAP

## note that BOOT partition is not the same like the others. Because in the new disk configuration they changed the boot partition.

# 5. create the software raid partitions

mdadm --create /dev/md0 --level=1 --raid-devices=2 --metadata=0.90 /dev/sdb1 missing

mdadm --create /dev/md1 --level=1 --raid-devices=2 --metadata=0.90 /dev/sdb2 missing

mdadm --create /dev/md2 --level=1 --raid-devices=2 --metadata=0.90 /dev/sdb3 missing

mdadm --create /dev/md3 --level=1 --raid-devices=2 --metadata=0.90 /dev/sdb4 missing

mdadm --create /dev/md4 --level=1 --raid-devices=2 --metadata=0.90 /dev/sdb5 missing

mdadm --create /dev/md5 --level=1 --raid-devices=2 --metadata=0.90 /dev/sdb6 missing

mkswap /dev/md5

# 6. copy the contents of / and /var/log directories to the new partitions

mkfs.ext3 /dev/md0

mkfs.ext3 /dev/md4

# 7. mount newly created/formatted partitions

mount /dev/md0 /mnt

mkdir -p /mnt/var/log

mount /dev/md4 /mnt/var/log

# 8. copy contents to the newly mounted directory

cp -xR --preserve=all / /mnt

# 9. create a mdadm file for boot process (!!!if you forget the file the MD devices will have different names)

### the head of the file should include these lines

echo "MAILADDR root" > /mnt/etc/mdadm.conf

echo "auto +imsm +1.x -all" >> /mnt/etc/mdadm.conf

echo "DEVICE /dev/sd*[a-z][1-9]" >> /mnt/etc/mdadm.conf

mdadm --detail --scan >> /mnt/etc/mdadm.conf

# 10. copy the contents to the root folder

cp /mnt/etc/mdadm.conf /etc

# 11. configure mount points

sed -i 's/LABEL=root-[a-zA-Z\-]*/\/dev\/md0/' /mnt/etc/fstab

sed -i 's/LABEL=swap-[a-zA-Z\-]*/\/dev\/md5/' /mnt/etc/fstab

sed -i 's/LABEL=logs-[a-zA-Z\-]*/\/dev\/md4/' /mnt/etc/fstab

sed -i '/md5/ a\/dev/md5 swap swap defaults 0 0 ' /mnt/etc/fstab

cp /mnt/etc/fstab /etc

# 12. change the label name for /dev/sdb1 partition

e2label /dev/sda1 |xargs -t e2label /dev/sdb1

# 13. bind mount dev sys proc to the mnt folder

mount --bind /dev /mnt/dev

mount --bind /sys /mnt/sys

mount --bind /proc /mnt/proc

chroot /mnt /bin/bash

# 14. install grub on /dev/sdb

grub-install /dev/sdb

# 15. backup initrd

cp /boot/initrd-$(uname -r).img /boot/initrd-$(uname -r).img.bck

# 16. create new initrd for raid

dracut --mdadmconf --fstab --add="mdraid" --filesystems "ext3 tmpfs devpts sysfs proc" --add-drivers="raid1 raid456 mdraid1x mdraid09" --force /boot/initrd-$(uname -r).img $(uname -r) -M

###never change the boot configuration via grub-mkconfig.. it will kill xenserver.. change the GRUB configuation, by hand inside the files

# 17. change grub configuration

sed -i 's/quiet/rd.auto rd.auto=1 rhgb quiet/' /boot/grub/grub.cfg

sed -i 's/LABEL=root-[a-zA-Z\-]*/\/dev\/md0/' /boot/grub/grub.cfg

sed -i '/search/ i\ insmod gzio' /boot/grub/grub.cfg

sed -i '/search/ i\ insmod part_msdos' /boot/grub/grub.cfg

sed -i '/search/ i\ insmod diskfilter mdraid09' /boot/grub/grub.cfg

sed -i '/search/ c\ set root=(hd0,gpt1)' /boot/grub/grub.cfg

# 18. exit from chroot

exit

# 19. change the same things in sda1 partition so after reboot you don't need to boot from second disk

cp /mnt/boot/initrd-3.10.0+10.img /boot/

sed -i 's/quiet/rd.auto rd.auto=1 rhgb quiet/' /boot/grub/grub.cfg

sed -i 's/LABEL=root-[a-zA-Z\-]*/\/dev\/md0/' /boot/grub/grub.cfg

sed -i '/search/ i\ insmod gzio' /boot/grub/grub.cfg

sed -i '/search/ i\ insmod part_msdos' /boot/grub/grub.cfg

sed -i '/search/ i\ insmod diskfilter mdraid09' /boot/grub/grub.cfg

sed -i '/search/ c\ set root=(hd0,gpt1)' /boot/grub/grub.cfg

# 20. reboot

# !!!!!!reboot the server the server will boot from software raid..!!!!!

# 21. After the reboot add the /dev/sda to the new MD disks.

sgdisk /dev/sdb -R /dev/sda

mdadm -a /dev/md0 /dev/sda1

mdadm -a /dev/md1 /dev/sda2

mdadm -a /dev/md2 /dev/sda3

mdadm -a /dev/md3 /dev/sda4

mdadm -a /dev/md4 /dev/sda5

# This will take a while for resync of all disks

grub-install /dev/sda

# Create SR

xe sr-create content-type=user device-config:device=/dev/md2 host-uuid=<host-uuid> name-label=”SRRaid1-Local” shared=false type=lvm

This post is basically just a backup of that forum post in case it become dead link

Letsencrypt kung-fu

May 4, 2017
computers, Linux, ...

Let’s encrypt CLI client is by far the most shittiest software ever invented, there is probably no doubt about it, but sadly, it’s the only interface that is supported, and unless you want to pay money for SSL certificate you need to live with that.

First of all – yes, their client (without asking or telling you) WILL run sudo and WILL use root and most likely WILL install garbage on your server that you don’t want to have there. If you never used letsencrypt client before, run it on testing VM first, before it desecrates your favorite web server with random garbage you don’t want there.

The letsencrypt client is written for dumb people, and it is based on undocumented black magic that I will try to uncover here a bit. The client basically works with a component called “certbot” which is a software that run on your server and does something to prove that you really own the domains for which you want to generate your SSL certificate. Because letsencrypt staff doesn’t want to bother you with technicalities they created this crap of a software to deal with them for you, in their own way, like it or not. It uses so called ACME (Automatic Certificate Management Environment) protocol to verify that you are owner. This thing is not a rocket science, and in a nutshell all it does is publish some data used to prove your ownership through your webserver, usually located on webroot/.well_known, their counter-party server will try to locate these by accessing your.domain/.well_known and in order to make it possible to verify your domain without modifications to your webserver, all you need to do is to create a central webroot and then make a symlink from all domain webroots to this one (just ln -s /var/www/letsencryptshite/.well_known /var/www/your.uber.tld/.well_known).

Once you do that, always pass these 2 parameters to their “software”:

--webroot --webroot-path /var/www/letsencrypt_shite

1	--webroot --webroot-path /var/www/letsencrypt_shite

I also strongly recommend you to maintain a comma separated list of all domains for which you want to get your certificate and store it somewhere like /etc/letsencrypt/domains because you will need to provide this list very often.

Now a little cheat sheet:

Renewing all domains

This can even be in your cron

./letsencrypt-auto renew --webroot --webroot-path /var/www/letsencrypt_shite

1	./letsencrypt-auto renew --webroot --webroot-path /var/www/letsencrypt_shite

You may need to restart / reload your web server after doing this, since the certificate will be overwritten, and Apache seems to be caching it somehow.

Adding or remove a domain and regenerate certificate

Modify your /etc/letsencrypt/domains list and run

./certbot-auto certonly --webroot --webroot-path /var/www/letsencrypt_shite/ --agree-tos --expand -d `cat /etc/letsencrypt/domains`

1	./certbot-auto certonly --webroot --webroot-path /var/www/letsencrypt_shite/ --agree-tos --expand -d `cat /etc/letsencrypt/domains`

Common locations:

/etc/letsencrypt – root of this thing’s config

/etc/letsencrypt/live – symlinks to current certificates, that’s where you can find chains for your domains

Example apache config that uses letsencrypt cert

<VirtualHost *:80>
    ServerName bena.rocks
    DocumentRoot /var/www/bena.rocks
</VirtualHost>

<VirtualHost *:443>
    SSLEngine on
    SSLCertificateFile /etc/letsencrypt/live/insw.cz/cert.pem
    SSLCertificateKeyFile /etc/letsencrypt/live/insw.cz/privkey.pem
    SSLCertificateChainFile /etc/letsencrypt/live/insw.cz/chain.pem
    ServerName bena.rocks
    DocumentRoot /var/www/bena.rocks
</VirtualHost>

ServerName bena.rocks

DocumentRoot /var/www/bena.rocks

</VirtualHost>

SSLEngine on

SSLCertificateFile /etc/letsencrypt/live/insw.cz/cert.pem

SSLCertificateKeyFile /etc/letsencrypt/live/insw.cz/privkey.pem

SSLCertificateChainFile /etc/letsencrypt/live/insw.cz/chain.pem

ServerName bena.rocks

DocumentRoot /var/www/bena.rocks

</VirtualHost>

Gentoo quick setup (for advanced gentoo users)

May 8, 2016
Linux

This is an excerpt from gentoo handbook containing only the stuff that really matters, with no extra stuff:

Prepare your disks

Do I need to explain how? 🙂 if yes, this is not for you

Mount them

mkdir /mnt/gentoo
mount root /mnt/gentoo
mount boot /mnt/gentoo/boot

mkdir /mnt/gentoo

mount root /mnt/gentoo

mount boot /mnt/gentoo/boot

Prepare stage3

cd /mnt/gentoo
lynx http://distfiles.gentoo.org/releases/amd64/autobuilds/current-install-amd64-minimal/
tar xvjpf stage3-*.tar.bz2 --xattrs

cd /mnt/gentoo

lynx http://distfiles.gentoo.org/releases/amd64/autobuilds/current-install-amd64-minimal/

tar xvjpf stage3-*.tar.bz2 --xattrs

Chroot

mirrorselect -i -o >> /mnt/gentoo/etc/portage/make.conf
mkdir /mnt/gentoo/etc/portage/repos.conf
cp /mnt/gentoo/usr/share/portage/config/repos.conf /mnt/gentoo/etc/portage/repos.conf/gentoo.conf
cp -L /etc/resolv.conf /mnt/gentoo/etc/
mount -t proc proc /mnt/gentoo/proc
mount --rbind /sys /mnt/gentoo/sys
mount --make-rslave /mnt/gentoo/sys
mount --rbind /dev /mnt/gentoo/dev
mount --make-rslave /mnt/gentoo/dev
chroot /mnt/gentoo /bin/bash
emerge-webrsync
emerge --sync

mirrorselect -i -o >> /mnt/gentoo/etc/portage/make.conf

mkdir /mnt/gentoo/etc/portage/repos.conf

cp /mnt/gentoo/usr/share/portage/config/repos.conf /mnt/gentoo/etc/portage/repos.conf/gentoo.conf

cp -L /etc/resolv.conf /mnt/gentoo/etc/

mount -t proc proc /mnt/gentoo/proc

mount --rbind /sys /mnt/gentoo/sys

mount --make-rslave /mnt/gentoo/sys

mount --rbind /dev /mnt/gentoo/dev

mount --make-rslave /mnt/gentoo/dev

chroot /mnt/gentoo /bin/bash

emerge-webrsync

emerge --sync

Emerge setup

eselect profile list
eselect profile set XXX
emerge --ask --update --deep --newuse @world
echo "Europe/Prague" > /etc/timezone
emerge --config sys-libs/timezone-data
emerge vim
vi /etc/locale.gen
locale-gen
eselect locale list
eselect locale set 5

eselect profile list

eselect profile set XXX

emerge --ask --update --deep --newuse @world

echo "Europe/Prague" > /etc/timezone

emerge --config sys-libs/timezone-data

emerge vim

vi /etc/locale.gen

locale-gen

eselect locale list

eselect locale set 5

Kernel

emerge sys-kernel/gentoo-sources sys-apps/pciutils 
cd /usr/src/linux 
# Build as you like
emerge <code>sys-kernel/linux-firmware

emerge sys-kernel/gentoo-sources sys-apps/pciutils

cd /usr/src/linux

# Build as you like

emerge <code>sys-kernel/linux-firmware

Initramfs

Pick one

# Genkernel
emerge sys-kernel/genkernel
genkernel --install initramfs

# Dracut
emerge dracut
cd /boot
dracut

# Genkernel

emerge sys-kernel/genkernel

genkernel --install initramfs

# Dracut

emerge dracut

cd /boot

dracut

Filesystems

Just edit /etc/fstab

/dev/sda2   /boot        ext2    defaults,noatime     0 2
/dev/sda3   none         swap    sw                   0 0
/dev/sda4   /            ext4    noatime              0 1

/dev/sda2 /boot ext2 defaults,noatime 0 2

/dev/sda3 none swap sw 0 0

/dev/sda4 / ext4 noatime 0 1

Networking

emerge net-misc/dhcpcd ntpd net-misc/netifrc

cd /etc/init.d
ln -s net.lo net.eth0
rc-update add net.eth0 default

emerge net-misc/dhcpcd ntpd net-misc/netifrc

cd /etc/init.d

ln -s net.lo net.eth0

rc-update add net.eth0 default

Grub

emerge --ask sys-boot/grub:2
grub2-install /dev/sda
grub2-mkconfig -o /boot/grub/grub.cfg

emerge --ask sys-boot/grub:2

grub2-install /dev/sda

grub2-mkconfig -o /boot/grub/grub.cfg

How to create login info in motd similar to ubuntu server

September 9, 2015
computers, Linux

Ever wondered how could you get this cool login screen you can see when you login to ubuntu server on other distros?

Welcome to Ubuntu 14.04.2 LTS (GNU/Linux 3.16.0-37-generic x86_64)

 * Documentation:  https://help.ubuntu.com/

  System information as of Tue Sep  8 14:10:39 CEST 2015

  System load:  0.14              Users logged in:     0
  Usage of /:   51.8% of 6.75GB   IP address for eth0: 192.168.2.1
  Memory usage: 27%               IP address for eth1: xxxxxxxxxxxx
  Swap usage:   10%               IP address for tun0: 192.168.8.1
  Processes:    136

  Graph this data and manage this system at:
    https://landscape.canonical.com/

Welcome to Ubuntu 14.04.2 LTS (GNU/Linux 3.16.0-37-generic x86_64)

* Documentation: https://help.ubuntu.com/

System information as of Tue Sep 8 14:10:39 CEST 2015

System load: 0.14 Users logged in: 0

Usage of /: 51.8% of 6.75GB IP address for eth0: 192.168.2.1

Memory usage: 27% IP address for eth1: xxxxxxxxxxxx

Swap usage: 10% IP address for tun0: 192.168.8.1

Processes: 136

Graph this data and manage this system at:

https://landscape.canonical.com/

This is a part of proprietary system called landscape. But this thing is too cool to remain proprietary, so I created a similar login info screen here: http://github.com/benapetr/system-info

All you need to do is to append it to your login scripts and here we go:

The programs included with the Debian GNU/Linux system are free software;
the exact distribution terms for each program are described in the
individual files in /usr/share/doc/*/copyright.

Debian GNU/Linux comes with ABSOLUTELY NO WARRANTY, to the extent
permitted by applicable law.
No mail.
Last login: Wed Sep  9 09:52:02 2015 from 27.23.broadband16.iol.cz

System information as of 2015-09-09 09:54:04:

  Users logged in: 2        System load: (0.22, 0.15, 0.08)
  Processes: 501            Free memory: 64%        
  Free swap: 96%

Generated by http://github.com/benapetr/system-info

The programs included with the Debian GNU/Linux system are free software;

the exact distribution terms for each program are described in the

individual files in /usr/share/doc/*/copyright.

Debian GNU/Linux comes with ABSOLUTELY NO WARRANTY, to the extent

permitted by applicable law.

No mail.

Last login: Wed Sep 9 09:52:02 2015 from 27.23.broadband16.iol.cz

System information as of 2015-09-09 09:54:04:

Users logged in: 2 System load: (0.22, 0.15, 0.08)

Processes: 501 Free memory: 64%

Free swap: 96%

Generated by http://github.com/benapetr/system-info

How to setup debian build environment for both amd64 and x86 platform

May 2, 2015
computers, Linux

This article describes how to setup environment in which you can build both amd64 as well x86 .deb packages. It’s assuming that you already have working debian on amd64 kernel.

First step: install debootstrap

sudo apt-get install debootstrap chroot
sudo mkdir /opt/jessie-386
sudo debootstrap --arch i386 jessie /opt/jessie-386/ http://ftp.debian.org/debian/

sudo apt-get install debootstrap chroot

sudo mkdir /opt/jessie-386

sudo debootstrap --arch i386 jessie /opt/jessie-386/ http://ftp.debian.org/debian/

Now you should have fully working x86 debian in /opt/jessie-386

Second step: make a script to switch into i386 version

Create a script anywhere you like, for example /bin/switch_32 with following content:

#!/bin/sh
r=/opt/jessie-386
sudo mount -o bind /proc $r/proc
sudo mount -o bind /dev $r/dev
sudo mount -o bind /sys $r/sys
sudo chroot $r
echo "Unmounting the filesystems from x86 system"
sudo umount $r/sys
sudo umount $r/proc
sudo umount $r/dev

#!/bin/sh

r=/opt/jessie-386

sudo mount -o bind /proc $r/proc

sudo mount -o bind /dev $r/dev

sudo mount -o bind /sys $r/sys

sudo chroot $r

echo "Unmounting the filesystems from x86 system"

sudo umount $r/sys

sudo umount $r/proc

sudo umount $r/dev

Then make it executable. Now running it would switch you into fully working debian x86 in which you can install all required packages using apt-get and build your x86 packages.

How to install mdadm on citrix xen 6.5

February 22, 2015
computers, Linux, ...
4 comments

For some reason citrix doesn’t like mdadm so they make everything possible to stop it from working on their xen server.

Here is a guide that would make it work there, but it may not survive system patching

Setup

Connect at least 2 disks to your box. Install a xen server without local storage on first disk.

Installing mdadm

The default install contains mdadm but it doesn’t load raid modules to kernel. In order to enable it, following needs to be done:

echo "modprobe raid1" > /etc/sysconfig/modules/raid.modules
modprobe raid1
chmod a+x /etc/sysconfig/modules/raid.modules

echo "modprobe raid1" > /etc/sysconfig/modules/raid.modules

modprobe raid1

chmod a+x /etc/sysconfig/modules/raid.modules

Partitioning the disks

Now we create a final schema we want to use on our server on disk /dev/sdb, xen needs to have at least 3 partitions, 1 is for boot loader, second is for OS, I recommend 20gb or more, because this disk is pretty much impossible to extend, although citrix defaults it to 4GB, last partition is for local storage and it should take all remaining space on disk.

Note: Citrix by default creates 3 partitions, 1 for OS, second is empty, same sized as first one and probably used for system upgrade. Third is used for local storage LVM. You don’t have to create second partition for it to work, but system upgrades may not be available if you don’t create it. On other hand system upgrades will likely not work anyway as citrix doesn’t support mdadm installations.

In this guide I will use old MS-DOS partition table because although it’s old, it’s much better supported and it just works. You can also use GPT partitions if you want, but I had some issues getting them work with mdadm and syslinux.

We will have a separate /boot partition for boot loader, because syslinux shipped with xen is having troubles booting from raid device for some reason.

So this is how the layout of sdb should look after we finish the partitioning:

/dev/sdb1 (2 GB) for bootloader
/dev/sdb2 (20 GB) for OS
/dev/sdb3 (rest) for LVM

# Configure disk
sgdisk --zap-all /dev/sdb
# Now run fdisk and create a new dos partition table
# make 2 partitions, one is for boot (/dev/sdb1 2gb) and one for Dom0 (/dev/sdb2 20gb)
#### after it's done create md device
mdadm --create /dev/md0 --level=1 --raid-devices=2 missing /dev/sdb2
mkfs.ext3 /dev/md0
mkfs.ext3 /dev/sdb1
mount /dev/md0 /mnt
cp -vxpR / /mnt
cd /mnt
mv boot /tmp/old_boot
mkdir boot
mount /dev/sdb1 /mnt/boot
mv /tmp/old_boot/* boot
# Fix /mnt/etc/fstab - replace LABEL with /dev/md0 and insert a record for /boot
# EXAMPLE:
head -2 /etc/fstab
/dev/md0    /         ext3     defaults   1  1 /dev/sda1   /boot     ext3     defaults   1  1  
# Update boot loader 
# You need to open /boot/extlinux.conf and replace all references to old disk with root=/dev/md0
mkdir /mnt/root/initrd-raid mkinitrd -v --fstab=/mnt/etc/fstab /mnt/root/initrd-raid/initrd-`uname -r`-raid.img `uname -r` cd /mnt/root/initrd-raid zcat initrd-`uname -r`-raid.img | cpio -i
mdadm --detail --scan >> etc/mdadm.conf
find . -print | cpio -o -Hnewc | gzip -c > /mnt/boot/initrd-`uname -r`-raid.img rm /mnt/boot/initrd-3.10-xen.img cd /mnt/boot ln -s initrd-`uname -r`-raid.img initrd-3.10-xen.img
extlinux -i boot/
cat /usr/share/syslinux/mbr.bin > /dev/sdb
# Open /mnt/boot/extlinux.conf
# remove absolute path to xen.gz to relative, /boot will be root device for bootloader (/xen.gz), replace LABEL with /dev/md0

###### example conf file for syslinux that works ######
# location mbr ui vesamenu.c32 serial 0 115200 default xe prompt 1 timeout 50

label xe
menu label XenServer
kernel mboot.c32
append xen.gz dom0_mem=752M,max:752M watchdog dom0_max_vcpus=2 crashkernel=128M@32M cpuid_mask_xsave_eax=0 console=vga vga=mode-0x0311 --- /boot/vmlinuz-3.10-xen root=/dev/md0 ro hpet=disable xencons=hvc console=hvc0 console=tty0 --- initrd-3.10-xen.img 
#######################EOF###################### 

reboot

# Configure disk

sgdisk --zap-all /dev/sdb

# Now run fdisk and create a new dos partition table

# make 2 partitions, one is for boot (/dev/sdb1 2gb) and one for Dom0 (/dev/sdb2 20gb)

#### after it's done create md device

mdadm --create /dev/md0 --level=1 --raid-devices=2 missing /dev/sdb2

mkfs.ext3 /dev/md0

mkfs.ext3 /dev/sdb1

mount /dev/md0 /mnt

cp -vxpR / /mnt

cd /mnt

mv boot /tmp/old_boot

mkdir boot

mount /dev/sdb1 /mnt/boot

mv /tmp/old_boot/* boot

# Fix /mnt/etc/fstab - replace LABEL with /dev/md0 and insert a record for /boot

# EXAMPLE:

head -2 /etc/fstab

/dev/md0 / ext3 defaults 1 1 /dev/sda1 /boot ext3 defaults 1 1

# Update boot loader

# You need to open /boot/extlinux.conf and replace all references to old disk with root=/dev/md0

mkdir /mnt/root/initrd-raid mkinitrd -v --fstab=/mnt/etc/fstab /mnt/root/initrd-raid/initrd-`uname -r`-raid.img `uname -r` cd /mnt/root/initrd-raid zcat initrd-`uname -r`-raid.img | cpio -i

mdadm --detail --scan >> etc/mdadm.conf

find . -print | cpio -o -Hnewc | gzip -c > /mnt/boot/initrd-`uname -r`-raid.img rm /mnt/boot/initrd-3.10-xen.img cd /mnt/boot ln -s initrd-`uname -r`-raid.img initrd-3.10-xen.img

extlinux -i boot/

cat /usr/share/syslinux/mbr.bin > /dev/sdb

# Open /mnt/boot/extlinux.conf

# remove absolute path to xen.gz to relative, /boot will be root device for bootloader (/xen.gz), replace LABEL with /dev/md0

###### example conf file for syslinux that works ######

# location mbr ui vesamenu.c32 serial 0 115200 default xe prompt 1 timeout 50

label xe

menu label XenServer

kernel mboot.c32

append xen.gz dom0_mem=752M,max:752M watchdog dom0_max_vcpus=2 crashkernel=128M@32M cpuid_mask_xsave_eax=0 console=vga vga=mode-0x0311 --- /boot/vmlinuz-3.10-xen root=/dev/md0 ro hpet=disable xencons=hvc console=hvc0 console=tty0 --- initrd-3.10-xen.img

#######################EOF######################

reboot

Now you should be able to boot from /dev/sdb if you are not there is something wrong with the setup, you need to figure out if your problem is with

MBR (No bootable device)
Boot loader (Missing operating system.)
/boot (Linux will start booting but die in progress – try removing quiet and splash from parameters)

Syncing the disks

Now if you were able to boot up you need to setup the sda disk

Create the same 3 partitions as you did on sdb on sda and then

dd if=/dev/sdb1 of=/dev/sda1
# mbr
cat /usr/share/syslinux/mbr.bin > /dev/sda
mdadm --add /dev/md0 /dev/sda2

Wait for disks to sync, meanwhile you can create a new local storage

mdadm --create /dev/md1 --level=1 --raid-devices=2 /dev/sda3 /dev/sdb3
 xe sr-create content-type=user device-config:device=/dev/md1 name-label="Local Storage" shared=false type=lvm

dd if=/dev/sdb1 of=/dev/sda1

# mbr

cat /usr/share/syslinux/mbr.bin > /dev/sda

mdadm --add /dev/md0 /dev/sda2

Wait for disks to sync, meanwhile you can create a new local storage

mdadm --create /dev/md1 --level=1 --raid-devices=2 /dev/sda3 /dev/sdb3

xe sr-create content-type=user device-config:device=/dev/md1 name-label="Local Storage" shared=false type=lvm

How to insert ubuntu PPA’s to debian

November 29, 2013
computers, Linux, ...

Ubuntu created a very interesting service called PPA – personal package archives.

If anyone of you ever used debian (or ubuntu) you may be wondering how cool it could be, if your software could be in official repository so that people could just type:

sudo apt-get install blah

These of you who managed to get through all the bureaucracy and got the packages there, might be wondering how cool it would be, if that package which got build and published there months ago, could be ever updated to latest version of your software 🙂

Today, it’s both possible and very easy, thanks to PPA.

I am not going to describe the process of how to submit stuff to PPA, because that is explained on many places, but I am going to explain how PPA work and how you can use this powerful service on debian.

What is PPA

PPA is basically a repository server, where each user can create their own personal aptitude remote repository, using unlimited number of own GPG keys. There is basically no difference between PPA repository and any other apt repository.

How do I make it work on debian

There is no command apt-add-repository on debian, because PPA is ubuntu thing. That makes 2 problems:

* You can’t easily add PPA
* You may have problems with dependencies, as some debian packages are named differently than ubuntu packages

First problem can be overriden easily. You can either get the source code of apt-add-repository and install it localy, OR you can just insert the url of PPA in format of

deb <a href="http://ppa.launchpad.net/benapetr/huggle/ubuntu">http://ppa.launchpad.net/USER/REPOSITORY/ubuntu</a> <span>YOUR_UBUNTU_VERSION_HERE</span> main&nbsp;

1	deb <a href="http://ppa.launchpad.net/benapetr/huggle/ubuntu">http://ppa.launchpad.net/USER/REPOSITORY/ubuntu</a> <span>YOUR_UBUNTU_VERSION_HERE</span> main

&nbsp;

to your /etc/apt/sources.list where USER is username on PPA and REPOSITORY is PPA. You also need to replace ubuntu version with any ubuntu version which is most close to your debian version, typically some LTS (precise, or lucid).

The other thing you need to do, is installation of GPG key of the user you want to download packages from:

gpg --keyserver <span><i>keyserver</i>.<i>ubuntu</i>.com </span>--recv-keys KEY

1	gpg --keyserver <span><i>keyserver</i>.<i>ubuntu</i>.com </span>--recv-keys KEY

gpg -a --export KEY | sudo apt-key add -

1	gpg -a --export KEY \| sudo apt-key add -

Now you should be able to download and use the PPA repository.

You may be however facing the other problem with dependecies, and there is currently no other solution for it, than rebuilding the source package yourself with correct debian dependencies.

Handling OOM issues gracefully on linux

June 27, 2013
computers, Linux

I figured out that there is almost no simple way to handle situation when your system is running out of memory. There is a subsystem called OOM killer implemented in the kernel, but that thing is truly dangerous. You probably never want to get your system to point when it is being used, because it might leave the system in unusable state.

For this reason I created a tool that allows everyone who uses linux to handle this kind of situation gracefully, it’s called terminator daemon, and it watches the system and eventually kills specific kinds of processes, which can be easily defined by an administrator.

You can find more about this tool on https://github.com/benapetr/terminator

Basic idea is, that when your system is running out of memory, this daemon will pick processes that can be “safely” killed without any impact on system and takes them out according to your settings. You can even prevent it from killing certain processes, for example: If you were running mysql server, you probably wouldn’t care about interactive shells being killed, but you would care if mysql was killed.

In such situation you can tell terminatord to never kill mysql, but other “user processes” can be safely killed.

In addition you can even set terminatord to execute some command on every kill, for example it can send you a mail that something wrong is going on with your system.

There are several examples on GitHub, here I copy pasted some of them:

Kill all processes that use more than 400mb of ram, except for user apache and root

# get uid of apache
grep apache /etc/passwd
# let's say apache is user 20, now we test it
terminatord -dvvv --soft 400 --hard 420 --ignore 20 --dry
# if everything is ok
terminatord -d --soft 400 --hard 420 --ignore 20 --kill

# get uid of apache

grep apache /etc/passwd

# let's say apache is user 20, now we test it

terminatord -dvvv --soft 400 --hard 420 --ignore 20 --dry

# if everything is ok

terminatord -d --soft 400 --hard 420 --ignore 20 --kill

Kill random processes in case that system has less than 100mb of free ram, except for root

# test it
terminatord -dvvv --ssoft 100 --shard 60 --quiet --dry

1 2	# test it terminatord -dvvv --ssoft 100 --shard 60 --quiet --dry

Combine both examples, in this example unlike the previous one, apache processes will not be killed, when system go OOM

# let's say apache is user 20, now we test it
terminatord -dvvv --soft 400 --hard 420 --ignore 20 --shard 60 --ssoft 100 --dry
# if everything is ok
terminatord -d --soft 400 --hard 420 --ignore 20 --kill --shard 60 --ssoft 100

# let's say apache is user 20, now we test it

terminatord -dvvv --soft 400 --hard 420 --ignore 20 --shard 60 --ssoft 100 --dry

# if everything is ok

terminatord -d --soft 400 --hard 420 --ignore 20 --kill --shard 60 --ssoft 100

Installing oracle 11i (12c) on debian sid (6.0)

June 24, 2013
computers, Linux, ...

Somehow it happened that I decided to try out oracle on latest linux kernel and debian system.

I horribly failed.

In order to make you not fail, here is a guide what to do in order to install it

1. Install necessary packages

apt-get install build-essential libaio1 gawk ksh libmotif3 alien libtool lsb-rpm make gcc libsc-dev libxp-dev

1	apt-get install build-essential libaio1 gawk ksh libmotif3 alien libtool lsb-rpm make gcc libsc-dev libxp-dev

This is probably not needed:
~~2. Update /dev/shm~~

Oracle requires shared pool to be present there, but latest linux is using /run/ to store it – this is dangerous and you should reboot your server once you finish (you only need to do that if /dev/shm points to /run/shm)

<strike>sudo rm /dev/shm</strike>

1	<strike>sudo rm /dev/shm</strike>

<strike>sudo mkdir /dev/shm</strike>

1	<strike>sudo mkdir /dev/shm</strike>

<strike>sudo chmod 1777 /dev/shm </strike>

1	<strike>sudo chmod 1777 /dev/shm </strike>

3. Change the system limits

Write the following options to the /etc/security/limits.confce file:

oracle soft nproc 2047        <br />oracle hard nproc 16383        <br />oracle soft nofile 1023        <br />oracle hard nofile 65535        <br />

1	oracle soft nproc 2047 <br />oracle hard nproc 16383 <br />oracle soft nofile 1023 <br />oracle hard nofile 65535 <br />

Add the following to /etc/pam.d/login to validate parameters /etc/security/limits.conf:

session required /lib/security/pam_limits.so        <br />session required pam_limits.so

1	session required /lib/security/pam_limits.so <br />session required pam_limits.so

Make sure you have this in your /etc/sysctl.conf file:

sysctl -p:         <br /># sysctl -p        <br />kernel.printk = 4 4 1 7        <br />kernel.maps_protect = 1        <br />fs.inotify.max_user_watches = 524288        <br />vm.mmap_min_addr = 65536        <br />net.ipv4.conf.default.rp_filter = 1        <br />net.ipv4.conf.all.rp_filter = 1        <br />fs.file-max = 65535        <br />kernel.shmall = 2097152        <br />kernel.shmmax = 2147483648        <br />kernel.shmmni = 4096        <br />kernel.sem = 250 32000 100 128        <br />net.ipv4.ip_local_port_range = 1024 65535        <br />net.core.rmem_default = 1048576        <br />net.core.rmem_max = 1048576        <br />net.core.wmem_default = 262144        <br />net.core.wmem_max = 262144

sysctl -p: # sysctl -p kernel.printk = 4 4 1 7 kernel.maps_protect = 1 fs.inotify.max_user_watches = 524288 vm.mmap_min_addr = 65536 net.ipv4.conf.default.rp_filter = 1 net.ipv4.conf.all.rp_filter = 1 fs.file-max = 65535 kernel.shmall = 2097152 kernel.shmmax = 2147483648 kernel.shmmni = 4096 kernel.sem = 250 32000 100 128 net.ipv4.ip_local_port_range = 1024 65535 net.core.rmem_default = 1048576 net.core.rmem_max = 1048576 net.core.wmem_default = 262144 net.core.wmem_max = 262144

4. Change some more links

ln -s /usr/bin/awk /bin/awk        <br />ln -s /usr/bin/rpm /bin/rpm        <br />ln -s /usr/bin/basename /bin/basename        <br />mkdir /etc/rc.d        <br />for i in 0 1 2 3 4 5 6 S ; do ln -s /etc/rc$i.d /etc/rc.d/rc$i.d ; done

1	ln -s /usr/bin/awk /bin/awk <br />ln -s /usr/bin/rpm /bin/rpm <br />ln -s /usr/bin/basename /bin/basename <br />mkdir /etc/rc.d <br />for i in 0 1 2 3 4 5 6 S ; do ln -s /etc/rc$i.d /etc/rc.d/rc$i.d ; done

5. Create user credentials

addgroup oinstall        <br />    Adding group 'oinstall' (GID 1001) ...        <br /> Done.        <br /># addgroup dba        <br /> Adding group 'dba' (GID 1002) ...        <br /> Done.        <br /># addgroup nobody        <br />    Adding group 'nobody' (GID 1003) ...        <br />    Done.        <br /># usermod -g nobody nobody

1	addgroup oinstall <br /> Adding group 'oinstall' (GID 1001) ... <br /> Done. <br /># addgroup dba <br /> Adding group 'dba' (GID 1002) ... <br /> Done. <br /># addgroup nobody <br /> Adding group 'nobody' (GID 1003) ... <br /> Done. <br /># usermod -g nobody nobody

&nbsp;

useradd -g oinstall -G dba -p password -d /home/oracle -s /bin/bash oracle

1	useradd -g oinstall -G dba -p password -d /home/oracle -s /bin/bash oracle

# passwd oracle        <br />    Password changed.        <br />mkdir /home/oracle&nbsp;

1	# passwd oracle <br /> Password changed. <br />mkdir /home/oracle

chown -R oracle:dba /home/oracle

1	chown -R oracle:dba /home/oracle

6. Change dash to bash

# if you see this:

1	# if you see this:

ls -l /bin/sh

1	ls -l /bin/sh

lrwxrwxrwx 1 root root 4 Mar  2 18:07 /bin/sh -&gt; dash

1	lrwxrwxrwx 1 root root 4 Mar 2 18:07 /bin/sh -> dash

# do this:

1	# do this:

sudo su

sudo su

cd /bin

cd /bin

rm sh

rm sh

ln -s bash sh

1	ln -s bash sh

&nbsp;

# These links needs to be created as well to fix problems with makefiles

1	# These links needs to be created as well to fix problems with makefiles

sudo su

sudo su

mkdir /usr/lib64

1	mkdir /usr/lib64

ln -s /lib/x86_64-linux-gnu/libgcc_s.so.1 /usr/lib64/<br />ln -s /usr/lib/x86_64-linux-gnu/libc_nonshared.a /usr/lib64/<br />ln -s /usr/lib/x86_64-linux-gnu/libpthread_nonshared.a /usr/lib64/<br />ln -s /usr/lib/x86_64-linux-gnu/libstdc++.so.6 /usr/lib64/

1	ln -s /lib/x86_64-linux-gnu/libgcc_s.so.1 /usr/lib64/<br />ln -s /usr/lib/x86_64-linux-gnu/libc_nonshared.a /usr/lib64/<br />ln -s /usr/lib/x86_64-linux-gnu/libpthread_nonshared.a /usr/lib64/<br />ln -s /usr/lib/x86_64-linux-gnu/libstdc++.so.6 /usr/lib64/

Now you can install oracle using their universal installer. If you don’t have GUI, you need to create a response file. Don’t forget to bypass system checks, because the version of libraries used by debian is too new for oracle

Petr Bena's Bl0g

Linux (9)

How to install mdadm to XenServer 7

Letsencrypt kung-fu

Renewing all domains

Adding or remove a domain and regenerate certificate

Common locations:

Example apache config that uses letsencrypt cert

Gentoo quick setup (for advanced gentoo users)

Prepare your disks

Mount them

Prepare stage3

Chroot

Emerge setup

Kernel

Initramfs

Filesystems

Networking

Grub

How to create login info in motd similar to ubuntu server

How to setup debian build environment for both amd64 and x86 platform

First step: install debootstrap

Second step: make a script to switch into i386 version

How to install mdadm on citrix xen 6.5

Setup

Installing mdadm

Partitioning the disks

Syncing the disks

How to insert ubuntu PPA’s to debian

What is PPA

How do I make it work on debian

Handling OOM issues gracefully on linux

Installing oracle 11i (12c) on debian sid (6.0)

Recent Posts

Recent Posts

Recent Comments

Archives

Categories

Meta