Difference between revisions of "Jontow Homelab KVM"

Revision as of 21:00, 16 October 2015

Ubuntu 15.04 Notes

Overview

This section should get a general build of a VM host running KVM with libvirt (virsh)
and 2TB of RAID1 (softraid) running on Ubuntu 15.04 server. The hardware platform
that this section targets/was tested on is a Sun Fire X2200 M2. There is a hardware
specific notes section below.

The disks will be setup with RAID and LVM, giving a maximum compromise of reliability
and flexibility. Swap partitions are setup outside of softraid to maximize performance.

Disk Config

(2) x 2.0 TB disks in chassis, built at install time using the following general procedure:
- From aux shell: fdisk /dev/sda and "g" to GPT the disk, repeat for /dev/sdb
- Create a new (bios_grub) partition on each disk: ~256MB is fine
- Create any swap partitions necessary
- Create a new identically sized partition on each disk, "Use as: physical volume for RAID"
- Configure Software RAID: RAID1, 2 devices, 0 spares, select the two identical partitions as members.
- Configure LVM:
  - Create VG with PV /dev/md0 (named "vm1-vg")
  - Create LV (20G) from VG "vm1-vg" (named "vm1-root")
  - Create LV (20G) from VG "vm1-vg" (named "vm1-iso-0")
  - Create LV (1T) from VG "vm1-vg" (named "vm1-store-0")
  - Setup "vm1-root" as / with ext4 FS
  - Setup "vm1-iso-0" as /vm1-iso with ext4 FS
  - Setup "vm1-store-0" as /vm1-store-0 with ext4 FS
- When asked, install grub to "/dev/sda" (it'll try to use "/dev/mapper", override this.)
- After firstboot of system, run "dpkg-reconfigure grub-pc" and choose to install on "/dev/sdb" as well.

Packages

 $ sudo apt-get install zsh multitail screen sysstat openssh-server nfs-kernel-server
 $ sudo apt-get install qemu-kvm libvirt-bin openvswitch-switch openvswitch-common
 $ sudo apt-get install libpolkit-agent-1-0 libpolkit-backend-1-0 policykit-1
 $ sudo apt-get install virtinst

System Config

/etc/network/interfaces :

# The loopback network interface
auto lo
iface lo inet loopback

################################################################################
# eth1: management interface (non-VM traffic)
auto eth1
iface eth1 inet static
        address 10.49.249.20
        netmask 255.255.255.0
        network 10.49.249.0
        broadcast 10.49.249.255
        gateway 10.49.249.1
        # dns-* options are implemented by the resolvconf package, if installed
        dns-nameservers 10.49.249.1
        dns-search je

################################################################################
# eth0: vlan-tagged for VM traffic
auto eth0
iface eth0 inet manual
################################################################################
# eth2: storage/cross-connect (xc) net, used with a pair of VM hosts
auto eth2
iface eth2 inet static
        address 10.49.254.20
        netmask 255.255.255.0
        network 10.49.254.0
        broadcast 10.49.254.255

/etc/hosts :

127.0.0.1       localhost
127.0.1.1       vm1

10.49.249.20    vm1.je
10.49.249.21    vm2.je

10.49.254.20    vm1.xc  vm1
10.49.254.21    vm2.xc  vm2

# The following lines are desirable for IPv6 capable hosts
::1     localhost ip6-localhost ip6-loopback
ff02::1 ip6-allnodes
ff02::2 ip6-allrouters

Setup storage repo:

 $ sudo ln -s /vm1-store-0 /vm-store-0

NFS

/etc/exports (on vm1):

 /vm1-store-0            vm2.xc(rw,sync,no_subtree_check,no_root_squash)

Enable/start NFS:

 $ sudo systemctl enable nfs-kernel-server
 $ sudo systemctl start nfs-kernel-server

Misc bugfixes / errors

"error: Diskfilter writes are not supported"

This occurs around bootup splash screen time, and does not affect system operation, but is ugly. Supposedly fixed in 15.10+.
Ugly quick fix:

 $ sudo vi /etc/grub.d/10_linux
 (Replace 'quick_boot="1"' with 'quick_boot="0"', then...)
 $ sudo update-grub

PCI-DMA: Out of IOMMU space for...

This SHOULD be platform-specific (Sun Fire X2200 M2). If you begin to see messages like this in your logs:

 Oct 16 18:04:55 vm1 kernel: [  113.785020] sata_nv 0000:00:05.0: PCI-DMA: Out of IOMMU space for 65536 bytes

You ought to consider the following fix:

 $ sudo vi /etc/default/grub

Add "iommu=memaper3" to this line:

 GRUB_CMDLINE_LINUX_DEFAULT="quiet splash"

Result should be:

 GRUB_CMDLINE_LINUX_DEFAULT="quiet splash iommu=memaper=3"

Requires a reboot.