Bhyve Hypervisor Freebsd ZFS


  • administrators

    Get updated to at least Freebsd 11.0

    Bhyve will function on previous releases after 10.2, maybe.


    Install bhyve and bhyve grub:

    pkg install vm-bhyve grub2-bhyve
    

    Load kernel modules:

    add these directives into /boot/loader.conf:

    if_bridge_load=”YES”
    if_tap_load=”YES”
    nmdm_load=”YES”
    vmm_load=”YES”
    

    We could load the modules manually, however I:undefined:’:undefined:ve had issues doing this the first time, but go ahead and try, maybe it will :undefined:‘:undefined:just work:undefined:’:undefined: for you.

    kldload if_bridge if_tap nmdm vmm
    

    When that does not work, (you might not even know at this point) just reboot and the modules will be loaded at that time.



    Add the following lines to /etc/rc.conf:

    vm_enable=”YES”
    vm_dir=”zfs:zroot/vms”
    vm_list=””
    vm_delay=”5″
    

    If you:undefined:’:undefined:ve not already setup storage, lets do it now. If you:undefined:’:undefined:ve not changed the default name for ZFS, it should be :undefined:‘:undefined:zroot:undefined:’:undefined:, find out by running:

    zfs list
    

    On this machine the output is:

    NAME                 USED  AVAIL  REFER  MOUNTPOINT
    bootpool             144M  1.72G   142M  /bootpool
    zroot               30.8G   860G    88K  /zroot
    zroot/ROOT          17.4G   860G    88K  none
    zroot/ROOT/default  17.4G   860G  17.4G  /
    zroot/tmp           40.3M   860G  40.3M  /tmp
    zroot/usr           11.6G   860G    88K  /usr
    zroot/usr/home      8.55G   860G  8.55G  /usr/home
    zroot/usr/ports     2.45G   860G  2.45G  /usr/ports
    zroot/usr/src        633M   860G   633M  /usr/src
    zroot/var           10.5M   860G    88K  /var
    zroot/var/audit       88K   860G    88K  /var/audit
    zroot/var/crash       88K   860G    88K  /var/crash
    zroot/var/log        692K   860G   692K  /var/log
    zroot/var/mail       244K   860G   244K  /var/mail
    zroot/var/tmp       9.33M   860G  9.33M  /var/tmp
    zroot/vms           1.62G   860G   500M  /vms
    zroot/vms/freebsd1  1.13G   860G  1.13G  /vms/freebsd1
    

    As we can see, :undefined:‘:undefined:zroot:undefined:’:undefined: is the pool name.!Important If the pool is named something other than :undefined:‘:undefined:zroot:undefined:’:undefined: , you must change the directives we did a minute ago within /etc/rc.conf to reflect the proper zpool.

    Create the storage:

    zfs create -o mountpoint=/vms zroot/vms
    

    Initialize:

    vm init
    

    If you get the error message"$vm_enable is not enabled in /etc/rc.conf!" ect:undefined:…:undefined: , this is due to the modules not being loaded as described earlier, just reboot and come back. Bookmark this page or check your history.

    Now we copy the templates to the /vms directory we:undefined:’:undefined:ve created:

    cp /usr/local/share/examples/vm-bhyve/* /vms/.templates/
    

    Creating the network bridge VMs will use:

    First lets find out what our inteface is called:

    ifconfig
    

    In this case the interface is called :undefined:‘:undefined:em0:undefined:’:undefined:

    So:

    vm switch create public
    
    vm switch add public em0
    

    Edit the template config files to use zvol instead of the IMG or iso You can use the iso, however we have ZFS for a reason don:undefined:’:undefined:t we.

    Templates are located where we copied them to earlier: /vms/.templates

    List all config files in the .templates directory:

    ls /vms/.templates
    

    Here are the default conf files listed:

    alpine.conf		centos7.conf		debian.conf		freebsd-zvol.conf	openbsd.conf		windows.conf
    centos6.conf		config.sample		default.conf		netbsd.conf		ubuntu.conf
    

    Have a look at the debian.conf template:

    loader="grub"
    cpu=1
    memory=512M
    network0_type="virtio-net"
    network0_switch="public"
    disk0_type="ahci-hd"
    disk0_name="disk0.img"
    grub_run_partition="1"
    grub_run_dir="/boot/grub"
    

    You can edit what you wish, but know what your editing, and what it will do.

    To make use of zvol we edit both :undefined:“:undefined:disk0_name:undefined:”:undefined: & :undefined:“:undefined:disk0_dev:undefined:”:undefined: from the above to look like this:

    loader="grub"
    cpu=1
    memory=512M
    network0_type="virtio-net"
    network0_switch="public"
    disk0_type="ahci-hd"
    disk0_name="disk0"
    disk0_dev="sparse-zvol"
    grub_run_partition="1"
    grub_run_dir="/boot/grub"
    

    Time to grab an iso, lets use freebsd: Head over tohttps://download.freebsd.org/ftp/releases/amd64/amd64/ISO-IMAGES/11.1/ and grab 11.1

    Copy the URL of the iso Running the following will snatch the image and deposit it locally.

    vm iso https://download.freebsd.org/ftp/releases/amd64/amd64/ISO-IMAGES/11.1/FreeBSD-11.1-RELEASE-amd64-disc1.iso
    

    Time to create the VM: This will create a VM of freebsd named :undefined:‘:undefined:freebsd11:undefined:’:undefined: with a disk size of ten gigs

    vm create -s 10G freebsd11
    

    Now install the image we just downloaded:

    vm -f install freebsd11 FreeBSD-11.1-RELEASE-amd64-disc1.iso
    

    You will be asked to select [vt100] for the console type, unless you know what:undefined:’:undefined:s up otherwise, choose it. Now continue with installation as you would normally.

    Once install is complete, log in and play, then type:

    poweroff
    

    List all VM:undefined:’:undefined:s:

    vm list
    
    NAME            DATASTORE       LOADER      CPU    MEMORY    VNC                  AUTOSTART    STATE
    freebsd11        default         bhyveload   1      256M      -                    No           Running (1923)
    

    Start the VM:

    vm start freebsd11
    

    To destroy the VM:

    vm destroy freebsd11
    

    To edit the config file for specific VM:

    vm configure freebsd11
    

    This will open the config file in your default editor.


  • administrators

    Alternatively or in addition to the above to install bhyve:

    pkg install vm-bhyve bhyve-firmware bhyve-rc-3 grub2-bhyve
    

    Output:

    To ensure binaries built with this toolchain find appropriate versions of the necessary run-time libraries, you may want to link using

    -Wl,-rpath=/usr/local/lib/gcc48

    For ports leveraging USE_GCC, USES=compiler, or USES=fortran this happens transparently.

    ===> NOTICE:

    This port is deprecated; you may wish to reconsider installing it:

    Unsupported by upstream. Use GCC 6 or newer instead:undefined:…:undefined: Message from vm-bhyve-1.1.8_1:

    To enable vm-bhyve, please add the following lines to /etc/rc.conf, depending on whether you are using ZFS storage or not. Please note that the directory or dataset specified should already exist.

    vm_enable="YES"
    vm_dir="zfs:pool/dataset"
    

    OR

    vm_enable="YES"
    vm_dir="/directory/path"
    

    Then run :

    vm init
    

    If upgrading from 1.0 or earlier, please note that the :undefined:‘:undefined:guest:undefined:’:undefined: configuration option is no longer used.

    Guests that are not using UEFI boot will need either loader=:undefined:“:undefined:grub:undefined:”:undefined: or loader=:undefined:“:undefined:bhyveload:undefined:”:undefined: in their configuration in order to make sure the correct loader is used.


    Message from bhyve-rc-3:

    Configuration is done completely though rc.conf. The rc script won:undefined:’:undefined:t touch any devices for you (neither disk, nor tap) so you need to make sure all of those have been initialized properly.

    General setup:

    kldload vmm
    net.link.tap.up_on_open=1
    

    Make it persistent:

    echo "net.link.tap.up_on_open=1" >> /etc/sysctl.conf
    cat >> /boot/loader.conf << EOF
    vmm_load="YES"
    EOF
    

    Minimal example:

    cat >> /etc/rc.conf << EOF
    cloned_interfaces="tap0 bridge0"
    bhyve_enable="YES"
    bhyve_diskdev="/dev/zvol/anything/bhyve/virt"
    EOF
    
    ifconfig tap0 create
    ifconfig bridge0 create
    
    service bhyve start
    tmux list-sessions
    tmux attach -t bhyve
    service bhyve status
    service bhyve stop
    

    Multi profile configuration example:

    cat >> /etc/rc.conf << EOF
    cloned_interfaces="tap0 tap1 bridge0"
    bhyve_enable="YES"
    bhyve_profiles="virt1 virt2"
    bhyve_virt1_diskdev="/dev/zvol/anything/bhyve/virt1"
    
    bhyve_virt2_tapdev="tap1"
    bhyve_virt2_diskdev="/dev/zvol/anything/bhyve/virt2"
    bhyve_virt2_memsize="8192"
    bhyve_virt2_ncpu="4"
    EOF
    
    ifconfig tap0 create
    ifconfig tap1 create
    ifconfig bridge0 create
    
    service bhyve start # start all
    service bhyve start virt2 # start individual
    tmux attach -t bhyve_virt1
    tmux attach -t bhyve_virt1
    service bhyve stop virt2 # stop individual
    service bhyve stop # stop all
    

    (by default ctrl-b d detaches from tmux).


 



  • Make ISO from DVD

    In this case I had an OS install disk which was required to be on a virtual node with no optical drive, so I needed to transfer an image to the server to create a VM

    Find out which device the DVD is:

    lsblk

    Output:

    NAME MAJ:MIN RM SIZE RO TYPE MOUNTPOINT sda 8:0 0 465.8G 0 disk ├─sda1 8:1 0 1G 0 part /boot └─sda2 8:2 0 464.8G 0 part ├─centos-root 253:0 0 50G 0 lvm / ├─centos-swap 253:1 0 11.8G 0 lvm [SWAP] └─centos-home 253:2 0 403G 0 lvm /home sdb 8:16 1 14.5G 0 disk /mnt sr0 11:0 1 4.1G 0 rom /run/media/rick/CCSA_X64FRE_EN-US_DV5

    Therefore /dev/sr0 is the location , or disk to be made into an ISO

    I prefer simplicity, and sometimes deal with the fallout after the fact, however Ive repeated this countless times with success.

    dd if=/dev/sr0 of=win10.iso

    Where if=Input file and of=output file

    I chill out and do something else while the image is being copied/created, and the final output:

    8555456+0 records in 8555456+0 records out 4380393472 bytes (4.4 GB) copied, 331.937 s, 13.2 MB/s

    Fin!

    read more
  • Recreate postrgresql database template encode to ASCII

    UPDATE pg_database SET datistemplate = FALSE WHERE datname = 'template1';

    Now we can drop it:

    DROP DATABASE template1;

    Create database from template0, with a new default encoding:

    CREATE DATABASE template1 WITH TEMPLATE = template0 ENCODING = 'UNICODE'; UPDATE pg_database SET datistemplate = TRUE WHERE datname = 'template1'; \c template1 VACUUM FREEZE;

    read more
});