Proxmox 3 Install at MCP

  • base netinst install of wheezy - deselect everything from tasksel
    • hostname testserver – domain mcp.lcl
    • default partition layout - noatime on root filesystem
    • for the purpose of running netinst, temporarily set vlan 1 to untagged for server trunks
    • Disk Partitioning:
      • SSD's in RAID-5 (castor/pollux) or RAID-1 (altair/deneb/vega)
      • RAID is PV for LVM
      • LVM → LV called “proxmox” (15 GB) – root fs, ext4, noatime
      • LVM → LV called “swap” (4 GB) – swap partition
      • LVM → LV called “vz” (60 GB) – ext4, mount on /mnt/ssd-vz (default mount options)
  • > /etc/motd
  • aptitude install ssh (this will pull in xauth, but that's okay)
  • aptitude install fbset screen rsync psmisc file patch ethtool strace tcpdump vim less
  • aptitude --without-recommends install dnsutils
  • adduser tdobes adm
  • sed -i -e 's/^deb.* main$/& contrib non-free/g' /etc/apt/sources.list
  • aptitude update && aptitude forget-new
  • aptitude install intel-microcode
  • aptitude --without-recommends install ntp
  • sed -i -e 's/^GRUB_CMDLINE_LINUX_DEFAULT="quiet"$/GRUB_CMDLINE_LINUX_DEFAULT="quiet panic=5"/g' /etc/default/grub && update-grub
  • aptitude --without-recommends install lvm2 mdadm bridge-utils ifenslave vlan # when asked about arrays to start at boot, enter root filesystem or leave blank for none; tell it yes to starting all arrays
  • aptitude --without-recommends install nut sshfs
  • aptitude install lsof memtest86+ parted time bzip2 sysstat
  • aptitude --without-recommends install cifs-utils
  • aptitude install unattended-upgrades
  • dpkg-reconfigure -plow unattended-upgrades # select “Yes”
  • sed -i -e 's/^127.0.1.1.*$/# &\n192.168.24.241\tcastor.mcp.lcl\tcastor\n192.168.24.242\tpollux.mcp.lcl\tpollux\n192.168.24.243\taltair.mcp.lcl\taltair\n192.168.24.244\tdeneb.mcp.lcl\tdeneb\n192.168.24.245\tvega.mcp.lcl\tvega\n192.168.24.240\ttestserver.mcp.lcl\ttestserver\tpvelocalhost/g' /etc/hosts
    
    echo >> /etc/network/interfaces
    echo '# PCIe' >> /etc/network/interfaces
    echo 'allow-hotplug eth2' >> /etc/network/interfaces
    echo 'iface eth2 inet manual' >> /etc/network/interfaces
    echo $'\tpre-up ifconfig $IFACE mtu 7200' >> /etc/network/interfaces
    echo >> /etc/network/interfaces
    echo '# server-to-server network' >> /etc/network/interfaces
    echo 'auto vmbr0' >> /etc/network/interfaces
    echo 'iface vmbr0 inet static' >> /etc/network/interfaces
    echo $'\taddress 192.168.24.240' >> /etc/network/interfaces
    echo $'\tnetmask 255.255.255.0' >> /etc/network/interfaces
    echo $'\tbridge_ports eth2' >> /etc/network/interfaces
    echo $'\tbridge_stp off' >> /etc/network/interfaces
    echo $'\tbridge_fd 0' >> /etc/network/interfaces
    echo $'\tmtu 7200' >> /etc/network/interfaces
    
    ifup eth2 && ifup vmbr0
  • echo '# PVE packages provided by proxmox.com' > /etc/apt/sources.list.d/pve.list
    echo deb http://download.proxmox.com/debian wheezy pve >> /etc/apt/sources.list.d/pve.list
    wget -O- "http://download.proxmox.com/debian/key.asc" | apt-key add -
    # new repos for pve post-3.1 (leave old 3.0/3.1 repo in place too)
    # see http://pve.proxmox.com/wiki/Package_repositories
    echo '# PVE pve-no-subscription repository provided by proxmox.com, NOT recommended for production use' > /etc/apt/sources.list.d/pve-no-subscription.list
    echo deb http://download.proxmox.com/debian wheezy pve-no-subscription >> /etc/apt/sources.list.d/pve-no-subscription.list
    aptitude update
    aptitude install postfix proxmox-ve-2.6.32 vzprocps open-iscsi bootlogd
    # on castor and pollux only - harms zfs arc performance, so don't use on altair, deneb or vega:
    aptitude install ksm-control-daemon
    # select "no configuration" for postfix
    aptitude full-upgrade
    # install the package maintainer's version of /etc/default/grub
    
    sed 's/^deb/# deb/g' -i /etc/apt/sources.list.d/pve-enterprise.list
    wget http://www.tolaris.com/blog/wp-content/uploads/2013/08/no_subscription_popup_pvemanagerlib.js-3.1.patch -O - | patch -p0
  • aptitude purge linux-image-amd64 linux-image-3.2.0-4-amd64 && aptitude install linux-base # tell it “No”, don't abort kernel removal
  • echo '# See /usr/share/postfix/main.cf.dist for a commented, more complete version' > /etc/postfix/main.cf
    echo >> /etc/postfix/main.cf
    echo 'smtpd_banner = $myhostname ESMTP $mail_name (Debian/GNU)' >> /etc/postfix/main.cf
    echo 'biff = no' >> /etc/postfix/main.cf
    echo >> /etc/postfix/main.cf
    echo "# appending .domain is the MUA's job." >> /etc/postfix/main.cf
    echo 'append_dot_mydomain = no' >> /etc/postfix/main.cf
    echo >> /etc/postfix/main.cf
    echo '# Uncomment the next line to generate "delayed mail" warnings' >> /etc/postfix/main.cf
    echo '#delay_warning_time = 4h' >> /etc/postfix/main.cf
    echo >> /etc/postfix/main.cf
    echo 'alias_maps = hash:/etc/aliases' >> /etc/postfix/main.cf
    echo 'alias_database = hash:/etc/aliases' >> /etc/postfix/main.cf
    echo 'mydestination = $myhostname, localhost.$mydomain, localhost' >> /etc/postfix/main.cf
    echo 'relayhost = ' >> /etc/postfix/main.cf
    echo 'mynetworks = 127.0.0.0/8' >> /etc/postfix/main.cf
    echo 'inet_interfaces = loopback-only' >> /etc/postfix/main.cf
    echo 'recipient_delimiter = +' >> /etc/postfix/main.cf
    echo >> /etc/postfix/main.cf
    echo >> /etc/postfix/main.cf
    /etc/init.d/postfix restart
  • # systemd doesn't work on the old 2.6 kernel used by proxmox 3.x # aptitude install systemd
  • # # ln -s `which systemd-journalctl` /usr/local/bin/journalctl
  • # sed -i -e 's|^GRUB_CMDLINE_LINUX_DEFAULT="quiet panic=5"$|GRUB_CMDLINE_LINUX_DEFAULT="quiet panic=5 init=/bin/systemd"|g' /etc/default/grub && update-grub
  • aptitude install nfs-kernel-server
  • on altair, deneb, vega: aptitude install zfsutils
  • reboot # to switch to pve kernel (so we can recover backed up files from ZFS)
  • # doesn't work, as init.d script overrides this on altair, deneb, vega: zpool export tank && zpool import -d /dev/disk/by-path/ tank
  • # copy in old /etc/ssh/ssh_host_* and /etc/network/interfaces
  • on vega: edit /etc/udev/rules.d/70-persistent-net.rules to ensure that eth2 is e1000e PCIe card and eth3/eth4 is igb
  • aptitude purge dbus
  • aptitude install irqbalance
  • reboot # to activate updated network config
  • on castor: pvecm create mcp
  • on pollux, altair, deneb, vega: pvecm add 192.168.24.241
  • on altair: echo $'\n/tank/software\t192.168.24.0/24(rw,async,no_subtree_check,no_root_squash)' >> /etc/exports && ln -s /tank/software /mnt/software && /etc/init.d/nfs-kernel-server start && exportfs -ra
  • on castor, pollux, deneb, vega: echo $'\n192.168.24.243:/tank/software\t/mnt/software\tnfs\ttcp,intr,_netdev\t0\t0' >> /etc/fstab && mkdir /mnt/software && mount /mnt/software
  • in proxmox web interface (on altair): datacenter → storage → add:
    • directory → “software” at /mnt/software/pve containing ISO, Templates - shared
    • LVM group → “ssd” on volume group “ssd” - not shared
    • directory → “ssd-vz” at /mnt/ssd-vz containing Containers - not shared
    • ZFS → “tank” on pool “tank”, limit nodes to altair,deneb,vega - thin provision (note that the pool will only appear if you're using webui from a node with the pool present)
  • on altair, deneb, vega: echo 'options zfs zfs_arc_max=25769803776' > /etc/modprobe.d/zfs.conf # (32 GB * 1024 * 1024 * 1024 * 0.75 == 75% of 32 GB RAM)
  • migrate back VM's which were set up previously… see also: http://pve.proxmox.com/wiki/Moving_disk_image_from_one_KVM_machine_to_another

See also: http://pve.proxmox.com/wiki/Storage:_ZFS

See also: http://pve.proxmox.com/wiki/Proxmox_VE_2.0_Cluster

See also: https://www.netlite.it/proxmox-tips/ http://www.jaxlug.net/wiki/2014/07/16

computer/mcp_proxmox_3_setup.txt · Last modified: 2015/06/23 02:02 by tdobes
Recent changes RSS feed Driven by DokuWiki Valid XHTML 1.0 Valid CSS