Backup Speed Comparison of Seagate Expansion 4TB Portable External Hard Drive USB 3.0 (STEA4000400) vs. Toshiba Canvio Basics 3TB Portable Hard Drive (HDTB330XK3CA)

IBS Review

Find this book at buch7.de | eurobuch.com | buchhandel.de | books.google.com ASIN=B017KE8OG0, Category: Hardware, Language: E

The 4TB Seagate STEA4000400 (sgt4tb) was delivered in 02-Apr-2016. Before first use, I wanted to write to all sectors of the drive and decided to use the (drive's internal) low level 'security erase' (function) of the 'hdparm' tool. It ran about 12 hours***. The CPU remains inactive during this operation. The smart data read error rate value remains unchanged.

Security erase writes to all sectors, not only those in use to reflect the reported capacity. It is even more low level than completely formatting the drive. (be careful not to interrupt this process otherwise the disk becomes unusable. low level erase only to be used by experts.)

I used hparam's secure erase to write to all sectors since with f3write the write speed goes down from an initial average above 60 MB/s to less than a tenth of that after few hundred GB had been written. The reason might be that f3write uses small write blocks. dd (disk-to-disk) writing /dev/zero or /dev/random also slowed down with sgt4tb after a while even when using large buffer size. I didn't care to research this further since neither f3write nor dd write all sectors anyway.

Here's the p (print) command output of the $ gdisk tool:

Disk /dev/sdb: 7814037167 sectors, 3.6 TiB
Logical sector size: 512 bytes
Disk identifier (GUID): FF8FF2AD-31F7-45B7-8D3F-463B660E0AA2
Partition table holds up to 128 entries
First usable sector is 34, last usable sector is 7814037133
Partitions will be aligned on 2048-sector boundaries
Total free space is 2014 sectors (1007.0 KiB)

Number Start (sector) End (sector) Size Code Name
1 2048 7814037133 3.6 TiB 8300 s4t1

$ sudo hdparm -tT /dev/sdb

/dev/sdb:
Timing cached reads: 2882 MB in 2.00 seconds = 1441.90 MB/sec
Timing buffered disk reads: 232 MB in 3.01 seconds = 76.98 MB/sec

the write speeds with 1 MiB buffer size:
168 MB/s $ dd if=/dev/zero of=/dev/sdb1 bs=1M count=1024! (1 GB data)
111 MB/s $ dd if=/dev/zero of=/dev/sdb1 bs=1M count=10240! (10 GB data)

Disk Encryption

The read/write benchmarks results obtained below are using a LUKS/LVM encrypted disk, similar to Full disk encryption with LUKS (including /boot) apart from using 'gdisk' instead of 'parted.' The latter can't create a GUID Partition Table (GPT) as required by LVM and for disk volumes above 2TB capacity. Though this is not a disk encryption guide, the following commands show which encryption method is used and the logical volumes layout.

The read/write speed for encrypted devices depends on both CPU speed and encryption method. It is adviced to chose the fastest (of the safest) of the latter according to results of cipher benchmark on your computer. ($ cryptsetup benchmark # requires the latest version of cryptsetup; the one in the ubuntu 12.04 LTS repository doesn't know action 'benchmark', see http://pastebin.com/9uUKricz or use arch linux instead of Ubuntu.)

[Update 09-Mar-2017: Links to installation guide added and command sequence amended]
Here's how the sgt4tb partitions (LVM on LUKS) were setup; it's just a sketch and not a guide for disk partitioning / formatting.

For more details on the latter, see e.g. Full disk encryption with LUKS (including /boot) by Pavel Kogan), GPT LUKS LVM Arch Install Guide 2016 by pavornoc, Minimal instructions for installing arch linux on an UEFI system (with full system encryption using dm-crypt and luks) by mattiaslundberg, and for non-uefi Install Arch Linux on Encrypted LVM by mattiaslundberg.

$ # Such large capacity devices require use of GPT fdisk
#
# Attention: replace 'sdb' and 'sdb1' below by the actual device name of your USB disk
$ gdisk /dev/sdb
# p Print current table
# o New GPT table
# y Confirm
# n New partition
# 1 Partition number
# [enter] accept default
# [enter] accept default
# w Write partition table
# y Confirm
# q Quit

$ cryptsetup luksFormat --hash=sha512 --key-size=256 --cipher=aes-xts-plain64 --verify-passphrase /dev/sdb1
$ cryptsetup luksOpen /dev/sdb1 lvm
$ pvcreate --dataalignment 1m /dev/mapper/lvm
$ vgcreate vg /dev/mapper/lvm
#
# We also create swap and root in case we want to make disk bootable later:
$ lvcreate -L 16G vg -n swap
$ lvcreate -L 32G vg -n root
#
$ # use 98% instead of 100% if you want room to provide for snapshots later
$ lvcreate -l +98%FREE vg -n home
#
$ mkfs.ext4 /dev/mapper/vg-root
$ mkswap -L swap /dev/mapper/vg-swap
$ mkfs.ext4 /dev/mapper/vg-home
#
# Linux reserves around 5% of the disk space to allow operations
# to continue when disk is full. This is only usefull for root partitions.
# So we claim to make available all free space for home partition:
$ tune2fs -m 0 /dev/mapper/vg-home
#
# prepare mounting home:
$ mkdir /media/sgt4tb
$ mount /dev/mapper/vg-home /media/sgt4tb

2 TB Data Backup Timing Comparison of 3TB Toshiba canvio basics (tcb3tb) vs. 4TB Seagate STEA4000400 (sgt4tb)

The data on the source drive WD my passport ultra 2tb (wdu2tb) consist of many tiny files, majority of files between 10 and 20 MB (jpg, orf(raw)), and a few big files up-to 80GB.

The backup with read verification command used by luckybackup is:

$ rsync -h --progress --stats -r -tgo -p -l -D --update --delete-after /media/wdu2tb /media/tcb3tb/wdu-blue/

$ rsync -h --progress --stats -r -tgo -p -l -D --update --delete-after /media/wdu2tb /media/sgt4tb/wdu-blue/

- tcp3tb: elapsed time : 15:43:58 (hh:mm:ss)
- sgt4tb: elapsed time : 35:01:15

Number of files: 382357
Number of files transferred: 375462
Total file size: 1919.00G bytes
Total transferred file size: 1919.00G bytes
Literal data: 1919.00G bytes
Matched data: 0 bytes

File list size: 7.86M
- tcb3tb: File list generation time: 1.908 seconds
- sgt4tb: File list generation time: 1.280 seconds
File list transfer time: 0.000 seconds
Total bytes sent: 1919.26G
Total bytes received: 7.15M

- tcb3tb: sent 1919.26G bytes received 7.15M bytes 33.89M bytes/sec
- sgt4tb: sent 1919.25G bytes received 7.15M bytes 15.23M bytes/sec
total size is 1919.00G speedup is 1.00

Backup Speed Comparison Result

The tcb3tb is more than twice as fast as sgt4tb (as measured connected to USB 3.0 adapter card; todo: test with T540p/USB 3.0 connectors).

The speed is not so important for backup drives. The most important is that data can safely be recovered.

When 800 GB were backup-ed from the tcb3tb to sgt4tb the former unexpectedly reported read errors for a few files, i.e. splicing 'splicing errors.' (in this case luckybackup's rsync reports: read ERRORs mapping "/media/sourcedrivename/path/filename": Input/output ERROR (5)) although the corresponding files were copied a few months ago with rsync with read verification to the tcb3tb! Worrying!

The sgt4tb internally uses Samsung Momentus 4TB SATA III 5400 RPM 2.5-inch Hard Drive (ST4000LM016). It is the first portable 4TB drive on the market. as it is using SMR (Shingled Magnetic Recording) technology to provide for higher density it is not as fast as regular (PMR) HDDs.

Like the 2TB Seagte backup plus 2tb slim, the sgt4tb allows to connect the internal drive directly to a sata port for data recovery in case the usb/sata adapter fails âS” or to replace the adapter and housing. (to remove the internal drive one has to break the case of corse.) (The tcb3tb also has an internal sata drive but it was especially conceived to work with their usb/sata adapter only. The WD my passport aka elements have non removable usb/sata adapter)

The sgt4tb support smart data** which can be read e.g. by the linux 'disk utility.' after a few hours of continous operations the "read error rate" value shows about (decimal) 250 million. The counter is 48 bit wide. after 2 days and more than 1TB transferred with rsync for backup (with f3write running when idle), the read error rate value shows 383 million. After backup of another 2TB, counter was at 640 million. Assessment is good, no sectors were remapped, no unrepairable errors occured. The counter's hex value indicates little endian binary representation.

[[Update 02-May-2016] The above backup test was done with the sgt4tb and default cryptsetup cypher (aes-cbc-essiv:sha256, Key: 256 bits). After reformatting the sgt4tb with

$ cryptsetup luksFormat --hash=sha512 --key-size=256 --cipher=aes-xts-plain64 --verify-passphrase /dev/sdx1

Again, the target sgt4tb was on USB 3.0 adapter and the source tcb3tb on USB 2.0 port. The backup comprises reading tcb3tb, writing sgt4tb, reading back sgt4tb for verification, and comparing checksum.

$ rsync -h --progress --stats -r -tgo -p -l -D --update --delete-after /media/tcb3tb/ /media/sgt4tb/

Number of files: 593619
Number of files transferred: 582302
Total file size: 2693.93G bytes
Total transferred file size: 2693.93G bytes
Literal data: 2693.93G bytes
Matched data: 0 bytes
File list size: 12.48M
File list generation time: 32.022 seconds
File list transfer time: 0.000 seconds
Total bytes sent: 2694.30G
Total bytes received: 11.10M

sent 2694.30G bytes  received 11.10M bytes  20.10M bytes/sec

total size is 2693.93G  speedup is 1.00

execution of task : tcb3tb --to-- sgt4tb, finished

Elapsed time : 24 + 13:13:54 = 37:13:54 = 134034 sec.
(checking transfer speed: 2694.30G * 1024 / 134034s = 20.58M bytes/sec)

Smart data of sgt4tb after this operation:
316 mio read error rate, lower 4*4bit zero
25 mio seek error rate
11.5 days power on
11.1 days head flying time

Verification Of Backup Data

$ cat rsync-compare.sh
SOURCE="/media/tcb3tb/wdu-blue/"
TARGET="/media/sgt4tb/wdu-blue/"
#
rsync -avnc $TARGET $SOURCE

$ date
Tue Apr 12 09:32:31 CEST 2016
$ ./rsync-compare.sh
sending incremental file list

.luckybackup-snaphots/
sent 13886185 bytes  received 6906 bytes  192.61 bytes/sec
total size is 1919001873179  speedup is 138126.34 (DRY RUN)
$ date
Wed Apr 13 05:34:44 CEST 2016

As expected only one file, i.e., '.luckybackup-snaphots/" was different.

(With source/target inversed '$ rsync -avnc $SOURCE $TARGET' ran slightly longer, i.e. 20h37m:

sent 13886611 bytes  received 6906 bytes  187.12 bytes/sec
total size is 1919001873179  speedup is 138122.11 (DRY RUN)

[[Update 20-May-2016] On Thinkpad T540p (i7-4700MQ CPU @ 2.40GHz quadcore) with both drives on USB 3.0 connectors the rsync-compare read speed is 57.4 MB/s which corresponds to a combined USB throughput of 114.8 MB/s. (sgt4tb and tcb3t both use cypher --hash=sha512 --key-size=256 --cipher=aes-xts-plain64)]

Notes

*) the disk is delivered with NTFS (for backward compatibilty). Since f3write (fff) showed very slow write rate, I changed the partition to linux ext4 file system which increased the speed at least 10 times. I didn't research the reason further (maybe the NTFS partition offset was badly aligned - which I doubt), since I was going to low level format the drive anyway and use one lvm/luks partition.

**) sadly neither the wdu2tb nor tcb3tb support 'smart data.'

***) Here is the sgt4tb data after security erase (same as reported before this operation):

****) The dual port USB 3.0 express card can't supply more than one external USB drives with power.

$ sudo hdparm -I /dev/sdb

/dev/sdb:

ATA device, with non-removable media
Model Number: ST4000LM016-1N2170
Serial Number: W800xxxx
Firmware Revision: 0003
Transport: Serial, ATA8-AST, SATA 1.0a, SATA II Extensions,
SATA Rev 2.5, SATA Rev 2.6, SATA Rev 3.0
Standards:
Used: unknown (minor revision code 0x001f)
Supported: 9 8 7 6 5
Likely used: 9
Configuration:
Logical max current
cylinders 16383 16383
heads 16 16
sectors/track 63 63
--
CHS current addressable sectors: 16514064
LBA user addressable sectors: 268435455
LBA48 user addressable sectors: 7814037168
Logical Sector size: 512 bytes
Physical Sector size: 4096 bytes
Logical Sector-0 offset: 0 bytes
device size with M = 1024*1024: 3815447 MBytes
device size with M = 1000*1000: 4000787 MBytes (4000 GB)
cache/buffer size = unknown
Form Factor: 2.5 inch
Nominal Media Rotation Rate: 5400
Capabilities:
LBA, IORDY(can be disabled)
Queue depth: 32
Standby timer values: spec'd by Standard, no device specific minimum
R/W multiple sector transfer: Max = 16 Current = 16
Advanced power management level: 128
Recommended acoustic management value: 208, current value: 0
DMA: mdma0 mdma1 mdma2 udma0 udma1 udma2 udma3 udma4 udma5 *udma6
Cycle time: min=120ns recommended=120ns
PIO: pio0 pio1 pio2 pio3 pio4
Cycle time: no flow control=120ns IORDY flow control=120ns
Commands/features:
Enabled Supported:
* SMART feature set
Security Mode feature set
* Power Management feature set
* Write cache
* Look-ahead
* Host Protected Area feature set
* WRITE_BUFFER command
* READ_BUFFER command
* NOP cmd
* DOWNLOAD_MICROCODE
* Advanced Power Management feature set
Power-Up In Standby feature set
* SET_FEATURES required to spinup after power up
SET_MAX security extension
* 48-bit Address feature set
* Device Configuration Overlay feature set
* Mandatory FLUSH_CACHE
* FLUSH_CACHE_EXT
* SMART error logging
* SMART self-test
* General Purpose Logging feature set
* WRITE_{DMA|MULTIPLE}_FUA_EXT
* 64-bit World wide name
* Write-Read-Verify feature set
* WRITE_UNCORRECTABLE_EXT command
* {READ,WRITE}_DMA_EXT_GPL commands
* Segmented DOWNLOAD_MICROCODE
* Gen1 signaling speed (1.5Gb/s)
* Gen2 signaling speed (3.0Gb/s)
* Gen3 signaling speed (6.0Gb/s)
* Native Command Queueing (NCQ)
* Host-initiated interface power management
* Phy event counters
* READ_LOG_DMA_EXT equivalent to READ_LOG_EXT
DMA Setup Auto-Activate optimization
Device-initiated interface power management
* Software settings preservation
* SMART Command Transport (SCT) feature set
* SCT Write Same (AC2)
* SCT Features Control (AC4)
* SCT Data Tables (AC5)
unknown 206[12] (vendor specific)
unknown 206[13] (vendor specific)
* DOWNLOAD MICROCODE DMA command
* WRITE BUFFER DMA command
* READ BUFFER DMA command
Security:
Master password revision code = 65534
supported
not enabled
not locked
not frozen
not expired: security count
supported: enhanced erase
Logical Unit WWN Device Identifier: 5000c50090232904
NAA : 5
IEEE OUI : 000c50
Unique ID : 090232904
Checksum: correct