Linux - Disks and Filesystems: Difference between revisions

From Smithnet Wiki
Jump to navigation Jump to search
 
 
(23 intermediate revisions by the same user not shown)
Line 1: Line 1:


== iSCSI ==
== BTRFS ==
 
See also [https://btrfs.wiki.kernel.org/index.php/SysadminGuide here]
 
Create a RAID5 array for data and metadata:
mkfs.btrfs -L data -d raid5 -m raid5 -f /dev/sdc /dev/sdd /dev/sde
 
View usage:
btrfs filesystem usage /data
 
Look for btrfs filesystems:
blkid --match-token TYPE=btrfs
 
=== Subvolumes ===
 
Create subvolume:
  btrfs subvolume create /data/db
 
Info:
btrfs subvolume list .
btrfs subvolume list /data
btrfs subvolume show /data/db
 
Delete subvolume:
  btrfs subvolume delete /data/db


* Block storage provider: iSCSI Target
Subvolumes can be mounted like separate filesystems
* Storage client: iSCSI Initiator
  mount -o subvol=/oldpath /dev/sda5 /newpath
* Dynamic Discovery: Initiator sends 'SendTargets' request to a single IP/port and if the target listens on multiple names and addresses, all of them are sent in a form of TargetName and TargetAddress (IP:port#).
so the subvolume is now also visible under newpath
* See [https://en.wikipedia.org/wiki/ISCSI here] for background and IQN naming.


=== Target ===
See also [https://fedoramagazine.org/working-with-btrfs-subvolumes/ here]


* Install package (and dependencies: targetcli
=== Compression ===
* Choose/create local area for disk images: /iscsi_disks


Start admin utility:
Mount with compression option in fstab:
  targetcli
  compress=zstd:1
/> cd /backstores/fileio
where the algorirm could also be lzo or zlib. Compression level can be increased to 2 or 3
/backstores/fileio> create disk01 /iscsi_disks/disk01.img 10G
/backstores/fileio> cd /iscsi
/iscsi> create iqn.2000-01.com.example:storage.target01
/iscsi> cd iqn.2000-01.com.example:storage.target01/tpg1/luns
/iscsi/iqn.20...t01/tpg1/luns> create /backstores/fileio/disk01
/iscsi/iqn.20...t00/tpg1/luns> cd ../acls
/iscsi/iqn.20...t00/tpg1/acls> create iqn.2000-01.com.example:initiator01
/iscsi/iqn.20...t00/tpg1/acls> cd iqn.2000-01.com.example:initiator01
/iscsi/iqn.20...an-server01> set auth userid=someuser
/iscsi/iqn.20...an-server01> set auth password=somepass
exit


Other commands within targetcli:
Per-file/directory/subvolume compression is also available:
  ls
  btrfs property get /somefile compression
  delete [object]
  btrfs property set /etc compression zlib
help


systemctl enable target
Degragment:
systemctl start target
  btrfs filesystem defragment -r /


If necessary, open firewall for 3260:
=== Quotas ===
firewall-cmd --add-service=iscsi-target --permanent
firewall-cmd --reload


See also [https://www.lisenet.com/2016/iscsi-target-and-initiator-configuration-on-rhel-7/ here]
See [https://btrfs.readthedocs.io/en/latest/Qgroups.html here].


=== Initiator ===
== Disk Management ==


Install package: iscsi-initiator-utils
=== LUKS Encryptioon ===


In /etc/iscsi/initiatorname.iscsi specify the iSCSI target:
Create an encrypted partition:
  InitiatorName=iqn.2000-01.com.example:initiator01
  cryptsetup --verify-passphrase luksFormat /dev/sdb1


In /etc/iscsi/iscsid.conf:
See also [https://docs.redhat.com/en/documentation/red_hat_enterprise_linux/6/html/security_guide/chap-security_guide-encryption#sect-Security_Guide-LUKS_Disk_Encryption-LUKS_Implementation here] for other cypher options.
node.session.auth.authmethod = CHAP
node.session.auth.username = username
node.session.auth.password = password


Discover target:
Open the device (to create "/dev/mapper/protected"):
  # iscsiadm -m discovery -t sendtargets -p san-server01
  cryptsetup luksOpen /dev/sdb1 protected
san-server01:3260,1 iqn.2000-01.com.example:storage.target01


More info:
Make filesystem:
  iscsiadm -m node -o show
  mkfs.ext4 /dev/mapper/protected
...


Login:
Emtry om /etc/fstab:
  iscsiadm -m node --login
  /dev/mapper/protected /mnt/disk2 ext4 defaults 1 2


Confirm session:
To allow partitions to be opened at boot:
  iscsiadm -m session -o show
  echo "datastore /dev/sdb1 none" >> /etc/crypttab


Confirm new device added (eg sdc):
On reboot, the password will be asked for to enable mounting.
cat /proc/partitions


Then, partition, format and mount /dev/sdc as normal.
=== General ===


Logout of iSCSI (after unmounting used filesystems):
Copy iso file to USB:
  iscsiadm -m node --logout
  lsblk
dd bs=4M if=somedisk.iso of=/dev/sdb conv=fdatasync status=progress


== Disk Management ==
Show all block devices, and LVMs:
lvmdiskscan


=== Grub ===
=== Grub ===
Line 130: Line 134:
  echo repair > /sys/block/md1/md/sync_action
  echo repair > /sys/block/md1/md/sync_action
  echo check > /sys/block/md1/md/sync_action
  echo check > /sys/block/md1/md/sync_action
=== Multipath ===
/sbin/mpconf --enable
Configuration:
* /etc/multipath.conf
systemctl enable multipathd
systemctl start multipathd
multipath -ll
MP definitions:
        multipath {
              wwid                    a000443abababfeeeccce32c5000003ca
              alias                  data1
        }
        multipath {
              wwid                    a000443abababfeeeccce32c5000003cb
              alias                  datak2
        }
plus a device definition connecting to a hardware device.
Remove warnings for non-MP disks:
blacklist {
    devnode "sda"
}


=== Partitioning ===
=== Partitioning ===
Line 141: Line 174:
Supports MBR and GPT
Supports MBR and GPT


See [https://www.gnu.org/software/parted/manual/html_chapter/parted_2.html#SEC8 Manual]
See [https://www.gnu.org/software/parted/manual/html_chapter/parted_2.html#SEC8 Manual] and [https://wiki.archlinux.org/title/Parted this].
 
parted /dev/sdb
 
Some commands:
 
(parted) print
(parted) print free
(parted) mklabel gpt
(parted) help mklabel
(parted) mkpart primary
(parted) mkpart logical
(parted) mkpart "Data Store" ext4 0% 100%
(parted) rm <partnum>
(parted) quit


=== LVM ===
=== LVM ===
Line 198: Line 245:
To alter the label:
To alter the label:
  e2label /dev/sda newlabel
  e2label /dev/sda newlabel
To get the UUID of the disk:
blkid


To mount at boot time, enter in
To mount at boot time, enter in
Line 205: Line 255:
  mkfs.xfs -L /home /dev/mapper/vg0-lv0
  mkfs.xfs -L /home /dev/mapper/vg0-lv0


=== BTRFS ===
== iSCSI ==
 
* Block storage provider: iSCSI Target
* Storage client: iSCSI Initiator
* Dynamic Discovery: Initiator sends  'SendTargets' request to a single IP/port and if the target listens on multiple names and addresses, all of them are sent in a form of TargetName and TargetAddress (IP:port#).
* See [https://en.wikipedia.org/wiki/ISCSI here] for background and IQN naming.
 
=== Target ===
 
* Install package (and dependencies: targetcli
* Choose/create local area for disk images: /iscsi_disks
 
Start admin utility:
targetcli
/> cd /backstores/fileio
/backstores/fileio> create disk01 /iscsi_disks/disk01.img 10G
/backstores/fileio> cd /iscsi
/iscsi> create iqn.2000-01.com.example:storage.target01
/iscsi> cd iqn.2000-01.com.example:storage.target01/tpg1/luns
/iscsi/iqn.20...t01/tpg1/luns> create /backstores/fileio/disk01
/iscsi/iqn.20...t00/tpg1/luns> cd ../acls
/iscsi/iqn.20...t00/tpg1/acls> create iqn.2000-01.com.example:initiator01
/iscsi/iqn.20...t00/tpg1/acls> cd iqn.2000-01.com.example:initiator01
/iscsi/iqn.20...an-server01> set auth userid=someuser
/iscsi/iqn.20...an-server01> set auth password=somepass
exit
 
Other commands within targetcli:
ls
delete [object]
help
 
systemctl enable target
systemctl start target


See also [https://btrfs.wiki.kernel.org/index.php/SysadminGuide here]
If necessary, open firewall for 3260:
firewall-cmd --add-service=iscsi-target --permanent
firewall-cmd --reload
 
See also [https://www.lisenet.com/2016/iscsi-target-and-initiator-configuration-on-rhel-7/ here]
 
=== Initiator ===
 
Install package: iscsi-initiator-utils
 
In /etc/iscsi/initiatorname.iscsi specify the iSCSI target:
InitiatorName=iqn.2000-01.com.example:initiator01
 
In /etc/iscsi/iscsid.conf:
node.session.auth.authmethod = CHAP
node.session.auth.username = username
node.session.auth.password = password
 
Discover target:
# iscsiadm -m discovery -t sendtargets -p san-server01
san-server01:3260,1 iqn.2000-01.com.example:storage.target01


Create a RAID5 array for data and metadata:
More info:
  mkfs.btrfs -L data -d raid5 -m raid5 -f /dev/sdc /dev/sdd /dev/sde
  iscsiadm -m node -o show
...


View usage:
Login:
  btrfs filesystem usage /data
  iscsiadm -m node --login


Look for btrfs filesystems:
Confirm session:
  blkid --match-token TYPE=btrfs
  iscsiadm -m session -o show


Create subvolume:
Confirm new device added (eg sdc):
  btrfs subvolume create /data/db
cat /proc/partitions


Info:
Then, partition, format and mount /dev/sdc as normal.
btrfs subvolume list /data
btrfs subvolume show /data/db


Delete subvolume:
Logout of iSCSI (after unmounting used filesystems):
  btrfs subvolume delete /data/db
iscsiadm -m node --logout


== Loopback Filesystem ==
== Loopback Filesystem ==
Line 243: Line 345:
  mkfs.xfs -L backups loopback.img
  mkfs.xfs -L backups loopback.img


And mount /dev/loop0 (-o loop) as a traditional device. Note: the losetup configuration is lost at restart so can't be added to /etc/fstab for at-boot mounting.
And mount /dev/loop0 (-o loop) as a traditional device. Note: the lo setup configuration is lost at restart so can't be added to /etc/fstab for at-boot mounting.


== Smarttools ==
== Smarttools ==
Line 249: Line 351:
/etc/smartmontools/smartd.conf
/etc/smartmontools/smartd.conf


Default to scan ATA/SCSI devices and report problems to root:
DEVICESCAN -H -m root -M exec /usr/libexec/smartmontools/smartdnotify -n standby,10,q
Or a specific device, and email an external user:
  /dev/sda -a -o on -S on -s (S/../.././02|L/../../6/03) -m [email protected]
  /dev/sda -a -o on -S on -s (S/../.././02|L/../../6/03) -m [email protected]
Scan for devices:
smartctl --scan
Show detailed information about a device:
smartctl --all /dev/sda

Latest revision as of 18:08, 12 July 2024

BTRFS

See also here

Create a RAID5 array for data and metadata:

mkfs.btrfs -L data -d raid5 -m raid5 -f /dev/sdc /dev/sdd /dev/sde

View usage:

btrfs filesystem usage /data

Look for btrfs filesystems:

blkid --match-token TYPE=btrfs

Subvolumes

Create subvolume:

 btrfs subvolume create /data/db

Info:

btrfs subvolume list .
btrfs subvolume list /data
btrfs subvolume show /data/db

Delete subvolume:

 btrfs subvolume delete /data/db

Subvolumes can be mounted like separate filesystems

mount -o subvol=/oldpath /dev/sda5 /newpath

so the subvolume is now also visible under newpath

See also here

Compression

Mount with compression option in fstab:

compress=zstd:1

where the algorirm could also be lzo or zlib. Compression level can be increased to 2 or 3

Per-file/directory/subvolume compression is also available:

btrfs property get /somefile compression
btrfs property set /etc compression zlib

Degragment:

 btrfs filesystem defragment -r /

Quotas

See here.

Disk Management

LUKS Encryptioon

Create an encrypted partition:

cryptsetup --verify-passphrase luksFormat /dev/sdb1

See also here for other cypher options.

Open the device (to create "/dev/mapper/protected"):

cryptsetup luksOpen /dev/sdb1 protected

Make filesystem:

mkfs.ext4 /dev/mapper/protected

Emtry om /etc/fstab:

/dev/mapper/protected /mnt/disk2 ext4 defaults 1 2

To allow partitions to be opened at boot:

echo "datastore /dev/sdb1 none" >> /etc/crypttab

On reboot, the password will be asked for to enable mounting.

General

Copy iso file to USB:

lsblk
dd bs=4M if=somedisk.iso of=/dev/sdb conv=fdatasync status=progress

Show all block devices, and LVMs:

lvmdiskscan

Grub

When installing on a RAID 1 mirror for the OS grub boot loader only installs on the first disk, so it that fails you can't boot off the second. To copy loader to the second disk:

grub> find /grub/stage1

This should find (hd0,0) and (hd1,0) which correspond for /dev/sda and /dev/sdb. Then temporarily make sdb the first disk and install:

device (hd0) /dev/sdb
root (hd0,0)
setup (hd0)

HD Parameters

Show settings/features:

hdparm -I /dev/sda

Test transfer rate:

hdparm -t --direct /dev/sda

Show power management setting:

hdparm -B /dev/sda

MD RAIDs

Create an array of 2 disks in a RAID1 (mirror):

mdadm --create /dev/md0 -l 1 -n 2 /dev/sdb1 /dev/sdc1

Monitor status with:

mdadm --detail /dev/md0
cat /proc/mdstat

Ensure RAID is detected at boot time:

mdadm -Es >> /etc/mdadm.conf

Remove a device from an array:

mdadm --remove /dev/md0 /dev/sdb1

Fail a drive in an array:

mdadm --fail /dev/md0 /dev/sdb1

Add a device to an array:

mdadm --add /dev/md0 /dev/sdb1

The /etc/cron.weekly/99-raid-check script can sometimes report:

WARNING: mismatch_cnt is not 0 on /dev/md1

The actual mismatch count can be found:

cat /sys/block/md1/md/mismatch_cnt

A repair and rebuild can be:

echo repair > /sys/block/md1/md/sync_action
echo check > /sys/block/md1/md/sync_action

Multipath

/sbin/mpconf --enable

Configuration:

  • /etc/multipath.conf
systemctl enable multipathd
systemctl start multipathd
multipath -ll

MP definitions:

       multipath {
              wwid                    a000443abababfeeeccce32c5000003ca
              alias                   data1
       }
       multipath {
              wwid                    a000443abababfeeeccce32c5000003cb
              alias                   datak2
       }

plus a device definition connecting to a hardware device.

Remove warnings for non-MP disks:

blacklist {
    devnode "sda"
}

Partitioning

FDisk

Supports MBR partitions

Parted

Supports MBR and GPT

See Manual and this.

parted /dev/sdb

Some commands:

(parted) print
(parted) print free
(parted) mklabel gpt
(parted) help mklabel
(parted) mkpart primary
(parted) mkpart logical
(parted) mkpart "Data Store" ext4 0% 100%
(parted) rm <partnum>
(parted) quit

LVM

Physical Volumes

To create a PV out of two partions:

pvcreate /dev/sdc1 /dev/sdd1

To show current PVs:

pvscan

Volume Groups

To create a VG:

vgcreate vg00 /dev/sd[cd]

To show all current VGs:

vgscan

To show details of a VG (including free PEs):

vgdisplay vg00

To extend a volume group by adding a new PV:

vgextend vg00 /dev/sde

To make a volume group available:

vgchange -ay vg00

Logical Volumes

To create a new LV:

lvcreate --size 100M vg00 -n lv00

or change --size option to --extents 500 or --extents 60%VG or -l 100%FREE

eg to create a RAID5 array out of 3 disks (2 data):

lvcreate -n lv00 --type raid5 -i 2 --extents 100%FREE vg00

Show status of LVM RAID:

lvs -a vg00

To rename a LV in VG vg01:

lvrename vg00 lvold lvnew

To remove a LV:

lvremove vg00/lv01

To show current LVs:

lvscan

Filesystems

To format with 1% minfree, large file support (see types in /etc/mke2fs.conf), journalling and a label:

mkfs.ext4 -m 1 -T largefile4 -j -L /home /dev/mapper/vg00-lv00

To alter the label:

e2label /dev/sda newlabel

To get the UUID of the disk:

blkid

To mount at boot time, enter in

  • /etc/fstab

Or to use XFS on a LV:

mkfs.xfs -L /home /dev/mapper/vg0-lv0

iSCSI

  • Block storage provider: iSCSI Target
  • Storage client: iSCSI Initiator
  • Dynamic Discovery: Initiator sends 'SendTargets' request to a single IP/port and if the target listens on multiple names and addresses, all of them are sent in a form of TargetName and TargetAddress (IP:port#).
  • See here for background and IQN naming.

Target

  • Install package (and dependencies: targetcli
  • Choose/create local area for disk images: /iscsi_disks

Start admin utility:

targetcli
/> cd /backstores/fileio
/backstores/fileio> create disk01 /iscsi_disks/disk01.img 10G
/backstores/fileio> cd /iscsi
/iscsi> create iqn.2000-01.com.example:storage.target01
/iscsi> cd iqn.2000-01.com.example:storage.target01/tpg1/luns
/iscsi/iqn.20...t01/tpg1/luns> create /backstores/fileio/disk01
/iscsi/iqn.20...t00/tpg1/luns> cd ../acls
/iscsi/iqn.20...t00/tpg1/acls> create iqn.2000-01.com.example:initiator01
/iscsi/iqn.20...t00/tpg1/acls> cd iqn.2000-01.com.example:initiator01
/iscsi/iqn.20...an-server01> set auth userid=someuser
/iscsi/iqn.20...an-server01> set auth password=somepass
exit

Other commands within targetcli:

ls
delete [object]
help
systemctl enable target
systemctl start target

If necessary, open firewall for 3260:

firewall-cmd --add-service=iscsi-target --permanent
firewall-cmd --reload

See also here

Initiator

Install package: iscsi-initiator-utils

In /etc/iscsi/initiatorname.iscsi specify the iSCSI target:

InitiatorName=iqn.2000-01.com.example:initiator01

In /etc/iscsi/iscsid.conf:

node.session.auth.authmethod = CHAP
node.session.auth.username = username
node.session.auth.password = password

Discover target:

# iscsiadm -m discovery -t sendtargets -p san-server01
san-server01:3260,1 iqn.2000-01.com.example:storage.target01

More info:

iscsiadm -m node -o show
...

Login:

iscsiadm -m node --login

Confirm session:

iscsiadm -m session -o show

Confirm new device added (eg sdc):

cat /proc/partitions

Then, partition, format and mount /dev/sdc as normal.

Logout of iSCSI (after unmounting used filesystems):

iscsiadm -m node --logout

Loopback Filesystem

dd if=/dev/zero of=loopback.img bs=1024M count=5
losetup -fP loopback.img

To show loopback device(s):

losetup -a
losetup -l

To delete loopback device:

losetup -d /dev/loop0

Then, create filesystem, eg:

mkfs.xfs -L backups loopback.img

And mount /dev/loop0 (-o loop) as a traditional device. Note: the lo setup configuration is lost at restart so can't be added to /etc/fstab for at-boot mounting.

Smarttools

/etc/smartmontools/smartd.conf

Default to scan ATA/SCSI devices and report problems to root:

DEVICESCAN -H -m root -M exec /usr/libexec/smartmontools/smartdnotify -n standby,10,q

Or a specific device, and email an external user:

/dev/sda -a -o on -S on -s (S/../.././02|L/../../6/03) -m [email protected]

Scan for devices:

smartctl --scan

Show detailed information about a device:

smartctl --all /dev/sda