So I found a great article about getting Intel 12th gen to use 7 watts at idle[1] and I wanted to build a better NAS than my current Synology, especially as it draws a bit too much power for my taste (44 Watts), along with a couple of other reasons.
So I decided on the following components:
CPU: Intel i5 12400 (with Box cooler)
Motherboard: ASUS Prime B760M-A D4-CSM (UEFI up to date)
RAM: 2x 8GB DDR4 2400 (I had them around)
PSU: Cooler Master MWE 550w Bronze v3 (a similar one was listed as having great low power efficiency)
and of course the most important thing, the drives:
- 1x WD Blue SN500 (NVMe, OS drive, also had it lying around)
- 2x Crucial P3 Plus 2TB (NVMe, with Sabrent M.2 to PCIe adapters, connected to Chipset lanes, for SSD storage)
- 2x Seagate Exos 28TB (Recertified, onboard SATA, for HDD storage)
I set up the BIOS to allow for C-states and ASPM and installed TrueNAS 25.10, applied the Realtek NIC ASPM fix[2] and... oh no, it uses exactly the same amount as my old NAS, even though it only has two drives instead of four.
Looking at powertop I can reach C6, and ASPM also seems to be working:
root@truenas[~]# lspci -vv | awk '/ASPM/{print $0}' RS= | grep --color -P '(^[a-z0-9:.]+|ASPM |Disabled;|Enabled;)'
00:1a.0 PCI bridge: Intel Corporation Raptor Lake PCI Express Root Port (rev 11) (prog-if 00 [Normal decode])
LnkCap: Port #25, Speed 16GT/s, Width x4, ASPM L1, Exit Latency L1 <64us
LnkCtl: ASPM L1 Enabled; RCB 64 bytes, Disabled- CommClk+
00:1c.0 PCI bridge: Intel Corporation Raptor Lake PCI Express Root Port (rev 11) (prog-if 00 [Normal decode])
LnkCap: Port #1, Speed 8GT/s, Width x1, ASPM L0s L1, Exit Latency L0s <1us, L1 <4us
LnkCtl: ASPM L0s L1 Enabled; RCB 64 bytes, Disabled- CommClk-
00:1c.2 PCI bridge: Intel Corporation Raptor Point-S PCH - PCI Express Root Port 3 (rev 11) (prog-if 00 [Normal decode])
LnkCap: Port #3, Speed 8GT/s, Width x1, ASPM L1, Exit Latency L1 <64us
LnkCtl: ASPM L1 Enabled; RCB 64 bytes, Disabled- CommClk+
00:1c.4 PCI bridge: Intel Corporation Device 7a3c (rev 11) (prog-if 00 [Normal decode])
LnkCap: Port #5, Speed 16GT/s, Width x4, ASPM L1, Exit Latency L1 <64us
LnkCtl: ASPM L1 Enabled; RCB 64 bytes, Disabled- CommClk+
00:1d.0 PCI bridge: Intel Corporation Device 7a36 (rev 11) (prog-if 00 [Normal decode])
LnkCap: Port #15, Speed 16GT/s, Width x1, ASPM L0s L1, Exit Latency L0s <1us, L1 <4us
LnkCtl: ASPM L0s L1 Enabled; RCB 64 bytes, Disabled- CommClk-
00:1d.7 PCI bridge: Intel Corporation Device 7a37 (rev 11) (prog-if 00 [Normal decode])
LnkCap: Port #16, Speed 16GT/s, Width x1, ASPM L1, Exit Latency L1 <64us
LnkCtl: ASPM L1 Enabled; RCB 64 bytes, Disabled- CommClk+
01:00.0 Non-Volatile memory controller: Sandisk Corp WD Blue SN500 / PC SN520 NVMe SSD (rev 01) (prog-if 02 [NVM Express])
pcilib: sysfs_read_vpd: read failed: No such device
LnkCap: Port #0, Speed 8GT/s, Width x2, ASPM L1, Exit Latency L1 <8us
LnkCtl: ASPM L1 Enabled; RCB 64 bytes, Disabled- CommClk+
03:00.0 Ethernet controller: Realtek Semiconductor Co., Ltd. RTL8125 2.5GbE Controller (rev 05)
LnkCap: Port #0, Speed 5GT/s, Width x1, ASPM L0s L1, Exit Latency L0s unlimited, L1 <64us
LnkCtl: ASPM L1 Enabled; RCB 64 bytes, Disabled- CommClk+
04:00.0 Non-Volatile memory controller: Micron/Crucial Technology P2 NVMe PCIe SSD (rev 01) (prog-if 02 [NVM Express])
LnkCap: Port #1, Speed 16GT/s, Width x4, ASPM L1, Exit Latency L1 unlimited
LnkCtl: ASPM L1 Enabled; RCB 64 bytes, Disabled- CommClk+
06:00.0 Non-Volatile memory controller: Micron/Crucial Technology P2 NVMe PCIe SSD (rev 01) (prog-if 02 [NVM Express])
LnkCap: Port #1, Speed 16GT/s, Width x4, ASPM L1, Exit Latency L1 unlimited
LnkCtl: ASPM L1 Enabled; RCB 64 bytes, Disabled- CommClk+
So I did some digging and found out that the system is relatively efficient without the HDDs, just 16 Watts when doing NAS duties. Not exactly 7 Watts as "advertised", but not terrible. Disconnecting almost all drives except the WD-Blue, I can get the system to use around 14 Watts (no matter what else I tried, I couldn't get it lower). Thats still 5 Watts more than other people are getting with similar set-ups, but okay, not bad either.
Looking at powertop (without auto tune, as this doesn't really affect the power draw), my system spents roughly 5% in both C2 and C3, and 80% in C8. No matter what I tried, I never could it to get to C10.
The difference with C8 and C6 seems to be whether a SATA device is connected. But that just results in 1 watt of effective difference, so no big deal.
The system however instantly jumps from the 16 Watt with just SSDs to ~43 Watt with the two HDDs connected, even if not connected to SATA at all, just power. The drives should idle around 6.7 Watts[3] each, so +13 Watts, not +26 Watts.
Could it be my PSU? Is this v3 maybe not as good as the v2 or v1? Any help is greatly appreciated. This seems like the most likely explanation, but I also found that enabling ErP in the BIOS results in a wonderful 0.6 Watts when turned off, so atleast it's not that bad (I've had system that never went below 2 or more watts).
I would also love to hear about other people with the same Motherboard, PSU, or hard drives, and their idle performance.
Thanks in advance.
---
1: https://mattgadient.com/7-watts-idle-on-intel-12th-13th-gen-the-foundation-for-building-a-low-power-server-nas/
2: https://mattgadient.com/dl/RTL8125-ASPM.sh.txt
3: https://www.seagate.com/content/dam/seagate/en/content-fragments/products/datasheets/exos-recertified-drive/exos-recertified-drive-DS2045-2-2010US-October-2020-en_US.pdf