The APM specification dating from 1992 includes some controls for hard drives, allowing a host system to specify the desired performance level of a disk and whether standby is permitted by sending commands to a disk. In addition to the above query types, SES also supports a number of commands, including activating the “locate” and “fault” LEDs if present, and the ability to individually power off drives. The first step is to map out the relationship between the physical chassis where the disks reside, and the logical devices enumerated by the operating system.
Once you’ve done so, you must test delivery to your “real” inbox—you don’t want to learn that delivery isn’t working after your storage has already become unavailable! If you’d feel safer with a team of experts monitoring your storage, consider a ZFS Support Subscription. If you rely on manually checking on your storage periodically, you will regret it. Another important aspect of managing your storage system is configuring notifications. Klara recommends embedding these details directly into the ZFS vdev properties of each disk—a feature Klara created, which will become generally available in the upcoming OpenZFS 2.2 release. In these configurations, your system may or may not support features like individual “locate” and “fault” LEDs.
For chassis with larger numbers of drives, or when connecting external JBOD chassis, it is common for the drives to connect to a specialized board that provides power and routing for the SATA/SAS signals to the controller. When building a storage system, there are many different ways the disks might be connected to the system. NVME-oF allows storage devices and arrays in remote chassis to be connected to local motherboards. NVMe storage comes in many form factors, from small M.2 devices to U.2 and other hot-swappable formats intended for servers. NVMe connects storage devices directly to the PCIe bus, offering extremely low latency and high throughput.
Most Seagate disks have configurable Extended Power Conditions (EPC) settings that include timers for how long the disk needs to stay idle before entering various low-power modes. Disk vendors typically provide their own vendor-specific ways to do persistent configuration of power management settings, so it’s worth trying to use those instead so the desired configuration doesn’t depend on the host system applying it, instead being configured in the drive (but in some cases it might be desirable to have the host configure that!). To prevent parking the heads at all a value greater than 128 may do the job (254 is a common choice, as the highest-power setting available), but it’s possible that some disks won’t behave this way because the ATA specification refers only to spinning down the disk and does not specify anything about parking heads. Typical SAS connectors support up to 4 drives per “lane”, but with an expander up to 255 devices are possible. An eight lane controller can only directly attach to 8 disks, requiring more controllers (consuming additional PCI-E slots) to connect more drives. This has long been the interface bus used by most home users to connect their hard drives, and is supported by nearly every motherboard.
I agree to receive your newsletters and accept the data privacy statement. Ensure device health & easy replacements with these valuable tips. Discover strategies to manage disk arrays on FreeBSD and related platforms/operating systems. Simply installing the apps and choosing a pool for k3s and docker creates a dataset and logs. Your pool gets writes from somewhere and ZFS is writing those to disk every 5 seconds.
Sounds like the drives being woken for the ZIL to flush writes to the ZFS pool and then going back to idle/sleep every 5 seconds. Enable the checkmark for the Syslog and choose a pool that is not based on hard drives. I had this same problem, using HGST data center refurb drives.
Unnamed devices can be specified by their specific SES device and element number. This greatly reduces the chance of getting it wrong when you (or the datacenter technician) physically pulls the disk. You can also reboot, and GEOM will pick up the multipath when it first tastes the disks during boot.
If you need more advanced functionality than mpsutil provides, LSI provides their native tools sas2ircu and sas3ircu for FreeBSD. On my system, this command produces a bright red LED lit for that slot, physically highlighting the correct drive to replace. So, to activate the LED for the first disk displayed above, we first need to determine the enclosure handle number (0001), and then the slot number of the disk (03). This partitions each disk and labels the ZFS partition with the enclosure, slot, and serial number of the corresponding disk. As with a number of tools in FreeBSD, sesutil supports outputting JSON via the libxo library.
In this case, there are at least two disks that I probably need to configure, since /dev/sde seems to be parking as often as about every 4 minutes (0.004 Hz) and /dev/sdc is only parking slightly less often. The smartmon_load_cycle_count_value metric seems like it would be the right one to query, but that actually expresses a percentage value (0-100) representing how many load cycles remain in the specified lifetime- on reaching 0 the disk has done a very large number of load cycles. It does support reading arbitrary metrics from text files written by other programs with its textfile collector however, which is fairly easy to integrate with arbitrary other tools. These communities are filled with knowledgeable individuals who can offer more personalized advice and help you navigate the complexities of long-term data storage.
The timer values specified are in milliseconds, so this example will park the disk heads after 30 minutes of inactivity. If we wanted to allow the disk to still park its heads but at minimum frequency, setting the APM value to 7Fh (hdparm -B 127) seems to be the correct choice. Of the three disks that I decided need some attention, I have one Western Digital disk and two Seagate ones.
The parking rate basically drops to zero at the time I updated the settings for the Seagate drives, and the Western Digital one hasn’t changed because it needs to be powered off to change that setting and I haven’t done so yet. The other slight annoyance when setting the idle3 timer on WD drives is that changes only take effect when the drive is powered on, usually meaning the host computer must be fully shut down and started back up for any changes to be seen- this makes experimentation to determine how raw timer values are interpreted a slower and more tedious process. Of particular note, WD Green drives ship configured to park the heads after only 8 seconds of inactivity which could notionally wear out the disk in a matter of months if the heads are cycling more-or-less continuously! For drives made by Western Digital, the inactivity timer for parking the heads is called the idle3 timer.
Using the no-op true command on reveryplay other paths to that disk, will cause GEOM to re-”taste” the disk and see the label and automatically add the additional paths to the existing multipath. This will write a GEOM Multipath label to the last sector of the disk. Each SAS Expander will present as a new /dev/ses# device, so your system may have more than one.
I moved the system dataset to the boot pool. I don’t move any data, no apps are running, this is a vanilla Scale install so far, yet the HDD is in constant work. 1 SSD to boot and 1 HDD to store data. Agree, I have used SeaChest with good results for this same issue on scale plus drive cache. If you do it on a live pool, I’d back up your data first.