No space left on device

Jan 24 07:14:12 MACHINE_NAME rsyslogd[953]: file '7' write error: No space left on device [v8.32.0 try http://www.rsyslog.com/e/2027]

Jan 15 12:39:03 MACHINE_NAME systemd-networkd-wait-online[29924]: Could not create manager: No space left on device

Jan 25 15:58:36 SERVER systemd-logind[876]: Failed to save user data /run/systemd/users/0: No space left on device

Sep 22 08:00:35 SERVER systemd[1]: rpm-ostreed.service: Failed to add control inotify watch descriptor for control group /system.slice/rpm-ostreed.service: No space left on device

Aug 25 19:58:00 SERVER_NAME dockerd[1322]: time="2020-08-25T19:58:00.476149096Z" level=error msg="Failed to log msg \"\" for logger json-file: write /srv/docker/containers/6f20b6203bd6872e25d5d61a64722cc4b4a550b9ac0bad5f178c1bb3f5/14e6676f20b6203bd6872e25d5d61a64722cc550b9ac0bad5f178c1bb3f5-json.log: no space left on device"

Sep 24 05:24:58 SERVER_NAME systemd[1]: rpm-ostreed.service: Failed to add control inotify watch descriptor for control group /system.slice/rpm-ostreed.service: No space left on device

sed: couldn't flush /var/run/samba/upgrades/sednY5V7A: No space left on device

PostgreSQL logs

ERROR:  could not extend file "base/16385/t17_174219": No space left on device

Related commands

• df -kh
• apt clean && apt autoclean && apt purge && apt autoremove
• snap remove, apt remove
• journalctl --disk-usage, journalctl --vacuum-size=200M
• journalctl -r | grep "No space left on device"
• SystemMaxUse of journal
• docker system df or docker system df --verbose
• docker system df -v | grep -A100 "VOLUME NAME"
• docker system prune or docker system prune -a or docker system prune --volumes
• docker buildx prune --keep-storage
• find . -ls 2>/dev/null | sort -rnk7 | more
• find -delete
• du -hs -- * 2>/dev/null
• ls -ltr | sort -rnk7
• ls -ld $(find ./) | sort -rnk5 | more
  • Argument list too long
• growpart, resize2fs
• sar -F --human
• macOS:
  • brew cleanup
  • ~/Library/Caches/
  • brew info
  • /cores/ folder, GrandPerspective

Related software errors

• Artifactory: 2020-10-05T08:37:25.312Z [jfrt ] [WARN ] [887efcf06c050966] [r.s.RepositoryServiceImpl:1822] [tp-nio-8081-exec-289] - 90% storage space reached. Your maximum limit is 93%. It is recommended to add disk space.
• Argument list too long

Directories, filesystems

• /var/log/
• /tmp/
• /proc/kcore
• /var/lib/snapd/cache/
• /var/cache/apt/archives/
• /var/lib/apt/
• /var/lib/snapd/snaps/
• /var/lib/apt/lists/
• /var/lib/prometheus/
• /swap.img
• /dev/sda1
• kubectl edit pvc
• pip cache purge

macOS

• du -hs /Users/
• du -hs /Users/*/brew/

Related

• Deallocate node
• Amazon Elastic Block Store (EBS): https://aws.amazon.com/premiumsupport/knowledge-center/ebs-volume-size-increase/
• MariaDB: 2023-03-30 5:06:42 4395974 [ERROR] mysqld: Disk full (/tmp/#sql_1b3c_0.MAI); waiting for someone to free some space... (errno: 28 "No space left on device")

See also

• File system, du, df, find, ls, mkdir, touch, locate
• systemd-journald: journalctl, /etc/systemd/journald.conf, journalctl logs, journalctl --list-boots, journalctl --disk-usage, journalctl -u kubelet, journalctl -u prometheus, journalctl --help
• Monitoring: On call, Monitoring software, Monitoring services, Resource monitoring, Metric colletion tools, network monitoring, SLA Management Monitoring Tools, Alarm/Alert, Resource starvation, Alerts and notifications, Monitoring Kubernetes, VictoriaMetrics, Sensu, LogicMonitor, Distributed tracing, Datadog Monitors
• du, du -hs, Disk space usage, df, ls, tree, find, docker system df, journalctl --disk-usage, No space left on device, ENOSPC, sar -F, growpart, resize2fs, ncdu, duf, dua-cli