NAS (TrueNAS)¶

Tunables¶

Note: these are geared towards NFS/SMB (i.e. net) performance with a VMXNET3 adapter. Note the nfsd.maxthreads is for a 96 core CPU.

System->Tunables !

truenas_tunables

cc_cubic_load, LOADER, YES
cc_dctcp_load, LOADER, YES
ern.ipc.nmbjumbop, SYSCTL, 262144
hint.isp.0.role, LOADER, 2
hint.isp.1.role, LOADER, 2
hint.isp.2.role, LOADER, 2
hint.isp.3.role, LOADER, 2
hw.pci.honor_msi_blacklist, LOADER, 0
hw.vmx.rxnqueue, LOADER, 32
hw.vmx.txnqueue, LOADER, 32
kern.ipc.maxsockbuf, SYSCTL, 16777216
kern.ipc.nmbclusters, SYSCTL, 262144
kern.ipc.nmbjumbo16, SYSCTL, 262144
kern.ipc.nmbjumbo9, SYSCTL, 262144
net.inet.ip.intr_queue_maxlen, SYSCTL, 4096
net.inet.tcp.cc.algorithm, SYSCTL, htcp
net.inet.tcp.delayed_ack, SYSCTL, 0
net.inet.tcp.fast_finwait2_recycle, SYSCTL, 1
net.inet.tcp.lro, SYSCTL, 1
net.inet.tcp.recvbuf_auto, SYSCTL, 1
net.inet.tcp.recvbuf_max, SYSCTL, 67108864
net.inet.tcp.recvspace, SYSCTL, 4192304
net.inet.tcp.rfc6675_pipe, SYSCTL, 1
net.inet.tcp.sack.enable, SYSCTL, 1
net.inet.tcp.sendbuf_auto, SYSCTL, 1
net.inet.tcp.sendbuf_inc, SYSCTL, 32768
net.inet.tcp.sendbuf_max, SYSCTL, 67108864
net.inet.tcp.sendspace, SYSCTL, 2097152
net.inet.tcp.slow_start_after_idle, SYSCTL, 0
net.inet.tcp.tcbhashsize, LOADER, 1048576
net.inet.tcp.timestamps, SYSCTL, 1
net.inet.tcp.tso, SYSCTL, 1
net.isr.bindthreads, SYSCTL, 1
net.isr.dispatch, SYSCTL, 1
net.isr.maxthreads, SYSCTL, -1
vfs.nfsd.maxthreads, SYSCTL, 128

https://www.truenas.com/community/resources/high-speed-networking-tuning-to-maximize-your-10g-25g-40g-networks.207/

Pool Creation¶

Options to note: 1MB block writes turn encryption on everywhere maybe cause you cant do it later turn on sync writes

NFS Shares¶

use NFS shares for linux clients make sure the share folder you want to use as your mount point has the same user:group settings as those you specify in truenas under advanced options when you create the nfs share

make the nfs mapall user root in truenas, leave everything else blank make sure to only allow read access (under advanced options in the truenas page)

NFS Service¶

Services->NFS

Enable NFSv4 Enable NFSv3 ownership model for NFSv4 (NFSv4 ownership model sucks and breaks a lot of my stuff). NFSv4 (4.2 I guess?) has parallel connection support, so the throughput is gonna be leagues better. UDP is not available in NFSv4 so don't bother checking "Serve UDP NFS clients"

SMB Shares¶

blank

Other¶

every week at 3am run a long smart test every sunday and thursday at 3am tun a short smart test run a scrub task every week enable jumbo frame

truenas if you really have to copy data around, and a lot of it, and fast (migration) REDACTED