Create bootable USB on Linux or OS x command line



  • To create a bootable image on a USB ‘stick’ aka ‘thumb drive’ ‘flash drive’ in linux or OS x we use the ‘dd’ command.

    In this case, we’ll write the latest (as of this writeup) Debian Image , which happens to be 8.5.0 amd64

    I generally try to use the torrent, taking in consideration the server/ bandwidth costs involved with free and open software.

    Download the image you wish to use. Many times the image you use, will depend on what size USB device you have available. Or obviously what you wish to accomplish.

    In this case I’m using a USB card reader, with a 32 GB Micro SD card


    You’ll need to know what USB device name to use. An easy way to view what devices are currently mounted would be to use:

    lsblk
    

    There are many options, or flags if you need them. From the man pages:

           -a, --all
                  Also list empty devices.  (By default they are skipped.)
    
           -b, --bytes
                  Print the SIZE column in bytes rather than in a human-readable
                  format.
    
           -D, --discard
                  Print information about the discarding capabilities (TRIM,
                  UNMAP) for each device.
    
           -d, --nodeps
                  Do not print holder devices or slaves.  For example, lsblk
                  --nodeps /dev/sda prints information about the sda device
                  only.
    
           -e, --exclude list
                  Exclude the devices specified by the comma-separated list of
                  major device numbers.  Note that RAM disks (major=1) are
                  excluded by default.  The filter is applied to the top-level
                  devices only.
    
           -f, --fs
                  Output info about filesystems.  This option is equivalent to
                  -o NAME,FSTYPE,LABEL,UUID,MOUNTPOINT.  The authoritative
                  information about filesystems and raids is provided by the
                  blkid(8) command.
    
           -h, --help
                  Display help text and exit.
    
           -I, --include list
                  Include devices specified by the comma-separated list of major
                  device numbers.  The filter is applied to the top-level
                  devices only.
    
           -i, --ascii
                  Use ASCII characters for tree formatting.
    
           -J, --json
                  Use JSON output format.
    
           -l, --list
                  Produce output in the form of a list.
    
           -m, --perms
                  Output info about device owner, group and mode.  This option
                  is equivalent to -o NAME,SIZE,OWNER,GROUP,MODE.
    
           -n, --noheadings
                  Do not print a header line.
    
           -o, --output list
                  Specify which output columns to print.  Use --help to get a
                  list of all supported columns.
    
                  The default list of columns may be extended if list is
                  specified in the format +list (e.g. lsblk -o +UUID).
    
           -O, --output-all
                  Output all available columns.
    
           -P, --pairs
                  Produce output in the form of key="value" pairs.  All
                  potentially unsafe characters are hex-escaped (\x<code>).
    
           -p, --paths
                  Print full device paths.
    
           -r, --raw
                  Produce output in raw format.  All potentially unsafe
                  characters are hex-escaped (\x<code>) in the NAME, KNAME,
                  LABEL, PARTLABEL and MOUNTPOINT columns.
    
           -S, --scsi
                  Output info about SCSI devices only.  All partitions, slaves
                  and holder devices are ignored.
    
           -s, --inverse
                  Print dependencies in inverse order.
    
           -t, --topology
                  Output info about block-device topology.  This option is
                  equivalent to -o NAME,ALIGNMENT,MIN-IO,OPT-IO,PHY-SEC,LOG-
                  SEC,ROTA,SCHED,RQ-SIZE,RA,WSAME.
    
           -V, --version
                  Display version information and exit.
    
           -x, --sort column
                  Sort output lines by column.
    

    In this case, the system spits out the following disks:

    NAME                        MAJ:MIN RM   SIZE RO TYPE   MOUNTPOINT
    sda                           8:0    0 465.8G  0 disk   
    └─isw_ccheigfjba_Volume0    254:0    0 931.5G  0 dmraid 
      ├─isw_ccheigfjba_Volume01 254:1    0 893.8G  0 dmraid /
      └─isw_ccheigfjba_Volume05 254:3    0  37.7G  0 dmraid 
    sdb                           8:16   0 465.8G  0 disk   
    └─isw_ccheigfjba_Volume0    254:0    0 931.5G  0 dmraid 
      ├─isw_ccheigfjba_Volume01 254:1    0 893.8G  0 dmraid /
      └─isw_ccheigfjba_Volume05 254:3    0  37.7G  0 dmraid 
    sdc                           8:32   1   1.9G  0 disk   
    ├─sdc1                        8:33   1   925M  0 part   
    └─sdc2                        8:34   1  86.1M  0 part
    

    We can see three devices on this system shown by ‘sda’ ‘sdb’ and ‘sdc’, where sda and sdb are each 500GB drives in a raid array. And sdc is the USB device we’ll use.


    These naming conventions will not always be the case, depending on operating system. sdc will not always be your USB device.


    It’s very important to insure you are writing to the correct disk, as well as not trying to write to a partition on the desired target disk. As in:

    sdc                           8:32   1   1.9G  0 disk   
    ├─sdc1                        8:33   1   925M  0 part   
    └─sdc2                        8:34   1  86.1M  0 part
    

    Where in the above example, we see ‘sdc’ as the disk, and two partitions called ‘sdc1’ and ‘sdc2’


    You could go about formatting the disk at this point, however the commands we will use in this example will simply ignore and over write any and all partitions, erasing any and all data on the target device. Effectively destroying anything currently on the device.


    Once the image, or ISO is downloaded to your local machine, navigate to the directory which your downloads reside (generally the users ‘Download’ directory).

    Of course it is not required to move to the directory where the image is, at that point you’ll need to type the path, I say go to the Downloads directory for easy of use and simplify the command.


    In this case the image name is: debian-8.5.0-amd64-DVD-1.iso

    The disk is: sdc

    So the command to write the image ‘debian-8.5.0-amd64-DVD-1.iso’ To disk ‘sdc’

    You may need to unmount the device before this will work

    umount /dev/sdc 
    

    You would then run the following command:

    sudo dd if=debian-8.5.0-amd64-DVD-1.iso of=/dev/sdc bs=1M
    

    You’ll of course be requested to type in your sudo password, or at least you should not be running as root to begin with.

    The write will commence, and depending on the machine capabilities and the size of the image being written, will dictate the speed at which the procedure takes.


    There is no progress bar with this command, so sit back, or get on with another task. This could take quite some time.


    Once the image is written to the USB device and completed, the output will look something to this effect:

    3808+0 records in
    3808+0 records out
    3992977408 bytes (4.0 GB) copied, 288.986 s, 13.8 MB/s
    

    We can see here, that the USB device is now partitioned and the ISO is available.

    sdc                           8:32   1  29.7G  0 disk   
    ├─sdc1                        8:33   1   3.7G  0 part   /media/rick/Debian 8.5.0 amd64 1
    └─sdc2                        8:34   1   416K  0 part
    

    There is quite a lot of discussion as to which block size to write, or as seen in the above command ‘bs=1M’ , I have used 5M, however much faster, I’ve had instances where {something @ somewhere} went sour and left me with a useless image. Possibly due to incorrect writes, that is however for another discussion.





Tmux Commands

screen and tmux

A comparison of the features (or more-so just a table of notes for accessing some of those features) for GNU screen and BSD-licensed tmux.

The formatting here is simple enough to understand (I would hope). ^ means ctrl+, so ^x is ctrl+x. M- means meta (generally left-alt or escape)+, so M-x is left-alt+x

It should be noted that this is no where near a full feature-set of either group. This - being a cheat-sheet - is just to point out the most very basic features to get you on the road.

Trust the developers and manpage writers more than me. This document is originally from 2009 when tmux was still new - since then both of these programs have had many updates and features added (not all of which have been dutifully noted here).

Action tmux screen
start a new session tmux OR
tmux new OR
tmux new-session
screen
re-attach a detached session tmux attach OR
tmux attach-session
screen-r
re-attach an attached session (detaching it from elsewhere) tmux attach -d OR
tmux attach-session -d
screen -dr
re-attach an attached session (keeping it attached elsewhere) tmux attach OR
tmux attach-session
screen -x
detach from currently attached session ^b d OR
^b :detach
^a ^d OR
^a :detach
rename-window to newname ^b , <newname> OR
^b :rename-window <newn>
^a A <newname>
list windows ^b w ^a w
list windows in chooseable menu ^a "
go to window # ^b # ^a #
go to last-active window ^b l ^a ^a
go to next window ^b n ^a n
go to previous window ^b p ^a p
see keybindings ^b ? ^a ?
list sessions ^b s OR
tmux ls OR
tmux list-sessions
screen -ls
toggle visual bell ^a ^g
create another window ^b c ^a c
exit current shell/window ^d ^d
split window/pane horizontally ^b " ^a S
split window/pane vertically ^b % ^a |
switch to other pane ^b o ^a <tab>
kill the current pane ^b x OR (logout/^D)
collapse the current pane/split (but leave processes running) ^a X
cycle location of panes ^b ^o
swap current pane with previous ^b {
swap current pane with next ^b }
show time ^b t
show numeric values of panes ^b q
toggle zoom-state of current pane (maximize/return current pane) ^b z
break the current pane out of its window (to form new window) ^b !
re-arrange current panels within same window (different layouts) ^b [space]
Kill the current window (and all panes within) ^b killw [target-window]
  • Again running smartctl after all is said and done:

    smartctl --all /dev/sda

    ddrescue-smartctl-after-rescue.png ddrescue-smartctl-2.png

    Yet an old drive in itself, I run the wheels off of them, and monitor regularly as anyone should.

    read more
  • Clone hard drive with ddrescue

    Testing a hard drive for failure via smartmontools

    Install smartmontools if not already present: (debian)

    apt-get install smartmontools

    Locate specific hard drives currently installed:

    lsblk

    In this case I choose device ‘sda’

    smartctl --all /dev/sda -q errorsonly

    Example of failing hard drive: smart-failure.png

    Preparing a used hard drive for cloning

    Clearing existing partitions from destination hard drive:

    In this case the destination drive is defined as sdc:

    I’ll use fdisk to clear all existing partitions:

    fdisk /dev/sdc Then print partition existing on sdc by typing the letter p

    p

    fdisk.png

    The above image shows some detail on the destination drive, including on FreeBSD partition named /dev/sdc4

    Since there is only one partition on the destination drive, we simply need to type the letter d at the fdisk command prompt:

    d

    fdisk-delete.png

    If there are more existing partitions on the destination drive, we are presented with them, and numbers to go along, we select which partition via numbers what we delete (in this case everything)

    fdisk-multiple-partitions.png

    Once we’ve removed the partitions, we type the letter w to write the changes, (actually delete the partitions)

    I type the letter p once again to make sure no partitions are still on the disk and then the letter w to write the changes:

    fdisk-write.png

    Using ddrescue to clone the hard drive:

    !Insure your drive letters are correct!

    The command I use to simplify the procedure including reading the damaged drive and putting as little pressure on it as we can manage, while getting as much data as we can, are as follows:

    Insure existing, or damaged hard drive first, and destination hard drive second (In other terms input file first and output file second)

    ddrescue -f -n /dev/sda /dev/sdc /root/recovery.log

    From https://linux.die.net/man/1/ddrescue

    -f, --force overwrite output device or partition

    -n, --no-split do not try to split or retry failed blocks

    We set a log file, which ddrescue can read later if for any reason the proceedure fails in the middle. Or also if we do not get as close to a 100% success rate when finished, the log file will set ddrescue to only attempt to get failed blocks on additional runs.

    ddrescue.png

    In this case 100% of the data was recovered, and transferred to the new hard drive:

    ddrescue-complete.png

    I run lsblk once again to check partitions have been transferred: ddrescue-new-partitions.png

    I then restart the machine:

    shutdown -r now

    After a restart, the journal will be recovered as well as other routine scripts run. I remove other old, damaged hard drives during the restart just before the BIOS kicks up.

    Again after a restart I run :

    lsblk

    ddrescue-new-harddrive.png

    Fin!

    Note This local server is a test environment running ProxMox. Everything running as it should after the clone.

    read more
});