Skip to content

ddrescue clone failing hard drive

  • Clone hard drive with ddrescue

    Testing a hard drive for failure via smartmontools

    Install smartmontools if not already present: (debian)

    apt-get install smartmontools

    Locate specific hard drives currently installed:

    lsblk


    In this case I choose device ‘sda’

    smartctl --all /dev/sda -q errorsonly

    Example of failing hard drive: smart-failure.png


    Preparing a used hard drive for cloning

    Clearing existing partitions from destination hard drive:

    In this case the destination drive is defined as sdc:

    I’ll use fdisk to clear all existing partitions:

    fdisk /dev/sdc Then print partition existing on sdc by typing the letter p

    p

    fdisk.png

    The above image shows some detail on the destination drive, including on FreeBSD partition named /dev/sdc4

    Since there is only one partition on the destination drive, we simply need to type the letter d at the fdisk command prompt:

    d

    fdisk-delete.png

    If there are more existing partitions on the destination drive, we are presented with them, and numbers to go along, we select which partition via numbers what we delete (in this case everything)

    fdisk-multiple-partitions.png

    Once we’ve removed the partitions, we type the letter w to write the changes, (actually delete the partitions)

    I type the letter p once again to make sure no partitions are still on the disk and then the letter w to write the changes:

    fdisk-write.png


    Using ddrescue to clone the hard drive:

    !Insure your drive letters are correct!

    The command I use to simplify the procedure including reading the damaged drive and putting as little pressure on it as we can manage, while getting as much data as we can, are as follows:

    Insure existing, or damaged hard drive first, and destination hard drive second (In other terms input file first and output file second)

    ddrescue -f -n /dev/sda /dev/sdc /root/recovery.log

    From https://linux.die.net/man/1/ddrescue

    -f, --force overwrite output device or partition

    -n, --no-split do not try to split or retry failed blocks

    We set a log file, which ddrescue can read later if for any reason the proceedure fails in the middle. Or also if we do not get as close to a 100% success rate when finished, the log file will set ddrescue to only attempt to get failed blocks on additional runs.

    ddrescue.png

    In this case 100% of the data was recovered, and transferred to the new hard drive:

    ddrescue-complete.png

    I run lsblk once again to check partitions have been transferred: ddrescue-new-partitions.png


    I then restart the machine:

    shutdown -r now

    After a restart, the journal will be recovered as well as other routine scripts run. I remove other old, damaged hard drives during the restart just before the BIOS kicks up.

    Again after a restart I run :

    lsblk

    ddrescue-new-harddrive.png

    Fin!


    Note This local server is a test environment running ProxMox. Everything running as it should after the clone.

    • 0 Votes
      1 Posts
      490 Views
      No one has replied
    • 0 Votes
      3 Posts
      1k Views
      rickR

      New script due to dependency changes:

      #! /bin/sh set -e

      if [ “$1” = “install” ] || [ “$1” = “upgrade” ]; then

      ACTION=$1

      elif [ “$1” = “” ]; then ACTION=install else echo “usage: $(basename $0) action [source]” echo “action is either install (default) or upgrade” echo “source is either tar (default) or git” exit 1 fi

      if [ “$2” = “git” ] || [ “$2” = “tar” ]; then SOURCE=$2 elif [ “$2” = “” ]; then SOURCE=tar else echo “Unknown source $2!” exit 1 fi

      if [ “$BPC_UID” ] && [ $(getent passwd “$BPC_UID” | cut -d ‘:’ -f 1) != “backuppc” ]; then echo “The uid = $BPC_UID is already in use!” exit 1 fi

      #Install dependencies

      apt-get -q update export DEBIAN_FRONTEND=noninteractive apt-get install -q -y apache2 apache2-utils libapache2-mod-perl2 par2 perl smbclient rsync tar gcc zlib1g zlib1g-dev rrdtool git make perl-doc libarchive-zip-perl libfile-listing-perl libxml-rss-perl libcgi-session-perl libacl1-dev wget iputils-ping pwgen

      #Set up backuppc user and directory

      if ! id backuppc >/dev/null 2>&1; then if [ “$BPC_UID” ]; then adduser --system --home /var/lib/backuppc --group --disabled-password --shell /bin/false --uid=“$BPC_UID” backuppc else adduser --system --home /var/lib/backuppc --group --disabled-password --shell /bin/false backuppc fi fi mkdir -p /var/lib/backuppc/.ssh chmod 700 /var/lib/backuppc/.ssh echo -e “BatchMode yes\nStrictHostKeyChecking no” > /var/lib/backuppc/.ssh/config if [ ! -e /var/lib/backuppc/.ssh/id_rsa ]; then ssh-keygen -q -t rsa -b 4096 -N ‘’ -C “BackupPC key” -f /var/lib/backuppc/.ssh/id_rsa fi chmod 600 /var/lib/backuppc/.ssh/id_rsa chmod 644 /var/lib/backuppc/.ssh/id_rsa.pub chown -R backuppc:backuppc /var/lib/backuppc/.ssh

      #Set password or read password file

      if [ “$BPC_PASS” ]; then PASSWORD=“$BPC_PASS” elif [ -e /root/password ]; then PASSWORD=$(cat /root/password) chmod 600 /root/password else PASSWORD=$(pwgen -s -1 32) echo “$PASSWORD” > /root/password chmod 600 /root/password fi echo “backuppc:$PASSWORD” | chpasswd backuppc

      #Get BackupPC release versions

      get_latest_release() { wget -q -O - “https://api.github.com/repos/$1/releases/latest” | grep ‘“tag_name”:’ | sed -E ‘s/.“([^”]+)"./\1/’
      } bpcver=$(get_latest_release “backuppc/backuppc”) bpcxsver=$(get_latest_release “backuppc/backuppc-xs”) rsyncbpcver=$(get_latest_release “backuppc/rsync-bpc”)

      mkdir -p /tmp/bpc cd /tmp/bpc

      if [ $SOURCE = “tar” ]; then

      ##Fetch and install latest stable releases

      wget https://github.com/backuppc/backuppc-xs/releases/download/$bpcxsver/BackupPC-XS-$bpcxsver.tar.gz wget https://github.com/backuppc/rsync-bpc/releases/download/$rsyncbpcver/rsync-bpc-$rsyncbpcver.tar.gz wget https://github.com/backuppc/backuppc/releases/download/$bpcver/BackupPC-$bpcver.tar.gz tar -zxf BackupPC-XS-$bpcxsver.tar.gz tar -zxf rsync-bpc-$rsyncbpcver.tar.gz tar -zxf BackupPC-$bpcver.tar.gz cd BackupPC-XS-$bpcxsver perl Makefile.PL make make test make install cd …/rsync-bpc-$rsyncbpcver ./configure make make install cd …/BackupPC-$bpcver fi

      if [ $SOURCE = “git” ]; then

      #Fetch and install the latest development code instead

      git clone https://github.com/backuppc/backuppc.git git clone https://github.com/backuppc/backuppc-xs.git git clone https://github.com/backuppc/rsync-bpc.git cd backuppc-xs perl Makefile.PL make make test make install cd …/rsync-bpc ./configure make make install cd …/backuppc ./makeDist --nosyntaxCheck --releasedate “date -u "+%d %b %Y"” --version ${bpcver}git tar -zxf dist/BackupPC-${bpcver}git.tar.gz cd BackupPC-${bpcver}git fi

      if [ $ACTION = “install” ]; then ./configure.pl --batch --cgi-dir /var/www/cgi-bin/BackupPC --data-dir /var/lib/backuppc --hostname backuppc --html-dir /var/www/html/BackupPC --html-dir-url /BackupPC --install-dir /usr/local/BackupPC fi

      if [ $ACTION = “upgrade” ]; then ./configure.pl --batch --config-path /etc/BackupPC/config.pl fi

      #Set up web server #Note that changing the apache user and group (/etc/apache2/envvars) could cause other services #provided by apache to fail. There are alternatives if you don’t want to change the apache #user: use SCGI or a setuid BackupPC_Admin script - see the docs. cp httpd/BackupPC.conf /etc/apache2/conf-available/backuppc.conf sed -i “/Require local/d” /etc/apache2/conf-available/backuppc.conf sed -i “s/export APACHE_RUN_USER=www-data/export APACHE_RUN_USER=backuppc/” /etc/apache2/envvars sed -i “s/export APACHE_RUN_GROUP=www-data/export APACHE_RUN_GROUP=backuppc/” /etc/apache2/envvars echo ‘’ > /var/www/html/index.html a2enconf backuppc a2enmod cgid service apache2 restart

      #Set up backuppc service

      cp systemd/init.d/debian-backuppc /etc/init.d/backuppc

      chmod 755 /etc/init.d/backuppc

      update-rc.d backuppc defaults

      chmod u-s /var/www/cgi-bin/BackupPC/BackupPC_Admin

      touch /etc/BackupPC/BackupPC.users

      sed -i “s/$Conf{CgiAdminUserGroup}.*/$Conf{CgiAdminUserGroup} = ‘backuppc’;/” /etc/BackupPC/config.pl

      sed -i “s/$Conf{CgiAdminUsers}.*/$Conf{CgiAdminUsers} = ‘backuppc’;/” /etc/BackupPC/config.pl

      chown -R backuppc:backuppc /etc/BackupPC

      echo $PASSWORD | htpasswd -i /etc/BackupPC/BackupPC.users backuppc

      service backuppc start

      #Clean up

      cd rm -rf /tmp/bpc echo “All done!” exit 0

    • 0 Votes
      2 Posts
      909 Views
      rickR

      For those of you who enjoy the excitement of running dangerous commands:

      Replace “.log” with the file extension you wish to remove

      find . -type f -name "*.log" -exec rm -f {} \;
    • 0 Votes
      1 Posts
      713 Views
      No one has replied
    • 0 Votes
      1 Posts
      679 Views
      No one has replied
    • 0 Votes
      1 Posts
      988 Views
      No one has replied
    • 1 Votes
      1 Posts
      959 Views
      No one has replied
    • Download a file wget

      Moved Linux Systems Guides
      1
      0 Votes
      1 Posts
      1k Views
      No one has replied
  • Again running smartctl after all is said and done:

    smartctl --all /dev/sda

    ddrescue-smartctl-after-rescue.png ddrescue-smartctl-2.png

    Yet an old drive in itself, I run the wheels off of them, and monitor regularly as anyone should.

FreeBSD Notes
  • rickR

    Screen recording can use webm as their format, it can be more simple to use a gif to embed into a website or forum, than adding scripts to host different video format.

    In this case I grabbed a screen record of the progress for writing zeros to a hard drive with dd

    Use ffmpeg to convert webm to gif:

    First create a pallet:

    Move into the directory which the webm is located, or type in the path

    Where ‘dd.webm’ is the screen recording

    ffmpeg -y -i dd.webm -vf palettegen palette.png

    Output:

    dd-ffmpeg.png

    Then convert the webm to gif:

    ffmpeg -y -i dd.webm -i palette.png -filter_complex paletteuse -r 10 dd.gif

    dd-webm-2.png

    This is what I ended up with, looks like any image of any alien on the interwebz, as if shot through a potato, some tweaking of the command is in my future. None the less.

    dd.gif

    read more

  • rickR

    Write zeros to all sectors

    Use the command ‘lsblk’ to find the drive you wish to erase

    I’m using ‘dd’ to erase things

    In this case I’m torching sdb

    Only use status=progress if you care it’s not necessary

    dd if=/dev/zero of=/dev/sdb bs=12M status=progress

    dd.gif

    read more

  • rickR

    Remove old kernel images that are cluttering the system

    zstd-no-space-error.png

    Most of these errors are due to low or not enough space left in the root partition.

    If you are using a Debian flavor:

    sudo apt-get autoremove --purge

    zstd-error.png

    Inform grub whenever an old kernel is removed:

    update grub

    update-grub.png

    Remove the un-used kernel config files:

    These will be the files pre cursed as ‘rc’ where installed kernels use ‘ii’

    … As well as files no longer used or required due to dependencies

    This command will detect, print, and remove left over cruft from previously installed packages or scripts, that have been removed or updated.

    sudo dpkg --purge $(dpkg -l | awk '/^rc/{print $2}')

    purge.png

    You can re-run the following to view the installed kernel(s):

    dpkg --list | grep linux-image

    installed-kernels.png

    read more

  • rickR

    To list all files in current directory including dot files (hidden files or directories), as well as print permissions :

    ls -la
    read more

  • rickR
    Install Ruby on Debian

    Install rbenv

    sudo apt install rbenv

    Then initialize the environment:

    rbenv init

    rbenv-init.png

    nano ~/.bashrc

    Type or paste the output above, Ctrl +x to save

    Apply:

    source .bashrc

    Install ruby-build

    Install git:

    sudo apt install git

    install-git.png

    mkdir -p "$(rbenv root)"/plugins

    Clone to local:

    git clone https://github.com/rbenv/ruby-build.git "$(rbenv root)"/plugins/ruby-build

    clone-ruby.png

    curl -fsSL https://github.com/rbenv/rbenv-installer/raw/main/bin/rbenv-doctor | bash

    ruby-doctor.png

    Now install Ruby:

    print ruby versions available

    rbenv install --list

    rb-list.png

    Install the latest version unless you have a reason otherwise:

    rbenv install 3.3.0

    On this system there was an error compiling at this point with fiddle, as well as psych: ruby-failed.png

    Therefore manually installing the following filled dependencies

    sudo apt install libtool sudo apt install libffi-dev

    rb-compile.png

    rbenv global 3.3.0

    print installed ruby version

    ruby -v

    ruby-version-installed.png

    read more