Difference between revisions of "No space left on device"
Jump to navigation
Jump to search
(136 intermediate revisions by 9 users not shown) | |||
Line 1: | Line 1: | ||
− | Jan 24 07:14:12 MACHINE_NAME [[rsyslogd]][953]: file '7' write error: [[No space left on device | + | Jan 24 07:14:12 MACHINE_NAME [[rsyslogd]][953]: file '7' write error: [[No space left]] on device [v8.32.0 try http://www.rsyslog.com/e/2027] |
+ | Jan 15 12:39:03 MACHINE_NAME [[systemd-networkd]]-wait-online[29924]: Could not create manager: No space left on device | ||
+ | |||
+ | Jan 25 15:58:36 SERVER systemd-logind[876]: Failed to save user data [[/run/]]systemd/users/0: [[No space left on device]] | ||
+ | |||
+ | Sep 22 08:00:35 SERVER systemd[1]: rpm-ostreed.service: Failed to add control [[inotify]] watch descriptor for control group /system.slice/[[rpm-ostreed]].service: No space left on device | ||
+ | |||
+ | Aug 25 19:58:00 SERVER_NAME dockerd[1322]: time="2020-08-25T19:58:00.476149096Z" level=error msg="Failed to log msg \"\" for logger [[json-file]]: write /srv/docker/containers/6f20b6203bd6872e25d5d61a64722cc4b4a550b9ac0bad5f178c1bb3f5/14e6676f20b6203bd6872e25d5d61a64722cc550b9ac0bad5f178c1bb3f5-json.log: [[no space left on device]]" | ||
+ | |||
+ | Sep 24 05:24:58 SERVER_NAME systemd[1]: [[rpm-ostreed]].service: Failed to add control [[inotify]] watch descriptor for control group /system.slice/rpm-ostreed.service: No space left on device | ||
+ | |||
+ | [[sed]]: couldn't flush [[/var/run/]]samba/upgrades/sednY5V7A: [[No space left on device]] | ||
+ | |||
+ | [[PostgreSQL logs]] | ||
+ | ERROR: could not extend file "base/16385/t17_174219": [[No space left on device]] | ||
+ | |||
+ | [[Mysql logs]] / [[MariaDB logs]] | ||
+ | 2023-03-30 5:06:42 4395974 [ERROR] mysqld: Disk full (/tmp/#sql_1b3c_0.MAI); waiting for someone to free some space... (errno: 28 "No space left on device") | ||
+ | |||
+ | == Related terms == | ||
+ | |||
+ | * <code>[[apt clean]]</code> && <code>[[apt autoclean]]</code> && <code>[[apt purge]]</code> && <code>[[apt autoremove]] && [[journalctl --vacuum-size]]=200M | grep "freed"</code> | ||
+ | * <code>[[snap remove]] yourpackage</code>, <code>[[apt remove]] yourpackage</code> | ||
+ | |||
+ | * <code>[[df -kh]]</code> | ||
+ | * <code>[[wajig large]]</code> | ||
+ | * <code>[[apt clean]]</code> && <code>[[apt autoclean]]</code> && <code>[[apt purge]]</code> && <code>[[apt autoremove]]</code> | ||
+ | * <code>[[journalctl --disk-usage]]</code>, <code>[[journalctl --vacuum-size]]=200M</code> | ||
+ | * <code>[[journalctl -r]] | grep "No space left on device"</code> | ||
+ | * <code>[[docker system df]]</code> or <code>[[docker system df --verbose]]</code> | ||
+ | * <code>[[docker system df]] -v | grep -A100 "VOLUME NAME"</code> | ||
+ | * <code>[[docker system prune]]</code> or <code>[[docker system prune -a]]</code> or <code>[[docker system prune --volumes]]</code> | ||
+ | * <code>[[docker buildx prune --keep-storage]]</code> | ||
+ | * <code>[[find .]] -ls 2>[[/dev/null]] | [[sort -rnk7]] | more</code> | ||
+ | * <code>[[find -delete]]</code> | ||
+ | * <code>[[du -hs -- *]] 2>/dev/null</code> | ||
+ | * <code>[[du -hs]] [[/var/log/nagios3/]] [[/var/log/sysstat/]] [[/var/lib/prometheus/]]</code> | ||
+ | * <code>ls -ltr | sort -rnk7</code> | ||
+ | * <code>[[ls]] -ld $(find ./) | [[sort -rnk5]] | more</code> | ||
+ | ** [[Argument list too long]] | ||
+ | * <code>[[growpart]], [[resize2fs]]</code> | ||
+ | * <code>[[sar -F --human]]</code> | ||
+ | * [[macOS]]: | ||
+ | ** <code>[[brew cleanup]]</code> | ||
+ | ** <code>[[~/Library/Caches/]]</code> | ||
+ | ** <code>[[brew info]]</code> | ||
+ | ** <code>[[/cores/]]</code> folder, [[GrandPerspective]] | ||
+ | * [[overlay2]] | ||
+ | * [[Nagios]]: [[SERVICE ALERT]]: localhost;Disk Space;[[CRITICAL]];SOFT;1;DISK CRITICAL | ||
+ | * <code>[[SystemMaxUse]]</code> of [[journal]] | ||
+ | * <code>[[imjournal.state.tmp]]</code> | ||
+ | |||
+ | * <code>[[~/.bitmonero/]]</code> | ||
+ | |||
+ | * <code>[[no space left on device: unknown]]</code> | ||
+ | |||
+ | == Related software errors == | ||
+ | * [[Artifactory]]: <code>2020-10-05T08:37:25.312Z [jfrt ] [WARN ] [887efcf06c050966] [r.s.RepositoryServiceImpl:1822] [tp-nio-8081-exec-289] - 90% storage space reached. Your maximum limit is 93%. It is recommended to add disk space.</code> | ||
+ | * [[Argument list too long]] | ||
+ | |||
+ | == Directories, filesystems == | ||
+ | * <code>[[/var/log/]]</code> | ||
+ | * <code>[[/tmp/]]</code> | ||
+ | * <code>[[/proc/kcore]]</code> | ||
+ | * <code>[[/var/lib/snapd/cache/]]</code> | ||
+ | * <code>[[/var/cache/apt/archives/]]</code> | ||
+ | * <code>[[/var/lib/apt/]]</code> | ||
+ | * <code>[[/var/lib/snapd/snaps/]]</code> | ||
+ | * <code>[[/var/lib/apt/lists/]]</code> | ||
+ | * <code>[[/var/lib/prometheus/]]</code> | ||
+ | * <code>[[/swap.img]]</code> | ||
+ | * <code>[[/dev/sda1]]</code> | ||
+ | * <code>[[kubectl edit pvc]]</code> | ||
+ | * <code>[[pip cache purge]]</code> | ||
+ | |||
+ | |||
+ | === [[macOS]] === | ||
+ | * <code>[[du -hs /Users/]]</code> | ||
+ | * <code>[[du -hs]] [[/Users/]]*[[/brew/]]</code> | ||
+ | * <code>du -hs /Users/*[[/Library/Containers/]]com.docker.docker 2>/dev/null</code> | ||
+ | * [[~/Library/Containers/com.docker.docker/Data/vms/0/data/Docker.raw]] | ||
+ | * [[GrandPerspective]] | ||
+ | |||
+ | == Related == | ||
+ | * [[Deallocate node]] | ||
+ | * [[Amazon Elastic Block Store (EBS)]]: https://aws.amazon.com/premiumsupport/knowledge-center/ebs-volume-size-increase/ | ||
+ | * [[MariaDB]]: <code>2023-03-30 5:06:42 4395974 [ERROR] mysqld: Disk full (/tmp/#sql_1b3c_0.MAI); waiting for someone to free some space... (errno: 28 "No space left on device")</code> | ||
== See also == | == See also == | ||
* {{Linux filesystem commands}} | * {{Linux filesystem commands}} | ||
* {{journalctl}} | * {{journalctl}} | ||
− | * | + | * {{monitoring}} |
+ | * {{du}} | ||
[[Category:Computing]] | [[Category:Computing]] | ||
[[Category:Storage]] | [[Category:Storage]] | ||
[[Category:Logging]] | [[Category:Logging]] |
Latest revision as of 20:28, 21 October 2024
Jan 24 07:14:12 MACHINE_NAME rsyslogd[953]: file '7' write error: No space left on device [v8.32.0 try http://www.rsyslog.com/e/2027]
Jan 15 12:39:03 MACHINE_NAME systemd-networkd-wait-online[29924]: Could not create manager: No space left on device
Jan 25 15:58:36 SERVER systemd-logind[876]: Failed to save user data /run/systemd/users/0: No space left on device
Sep 22 08:00:35 SERVER systemd[1]: rpm-ostreed.service: Failed to add control inotify watch descriptor for control group /system.slice/rpm-ostreed.service: No space left on device Aug 25 19:58:00 SERVER_NAME dockerd[1322]: time="2020-08-25T19:58:00.476149096Z" level=error msg="Failed to log msg \"\" for logger json-file: write /srv/docker/containers/6f20b6203bd6872e25d5d61a64722cc4b4a550b9ac0bad5f178c1bb3f5/14e6676f20b6203bd6872e25d5d61a64722cc550b9ac0bad5f178c1bb3f5-json.log: no space left on device"
Sep 24 05:24:58 SERVER_NAME systemd[1]: rpm-ostreed.service: Failed to add control inotify watch descriptor for control group /system.slice/rpm-ostreed.service: No space left on device
sed: couldn't flush /var/run/samba/upgrades/sednY5V7A: No space left on device
ERROR: could not extend file "base/16385/t17_174219": No space left on device
2023-03-30 5:06:42 4395974 [ERROR] mysqld: Disk full (/tmp/#sql_1b3c_0.MAI); waiting for someone to free some space... (errno: 28 "No space left on device")
Contents
Related terms[edit]
apt clean
&&apt autoclean
&&apt purge
&&apt autoremove && journalctl --vacuum-size=200M | grep "freed"
snap remove yourpackage
,apt remove yourpackage
df -kh
wajig large
apt clean
&&apt autoclean
&&apt purge
&&apt autoremove
journalctl --disk-usage
,journalctl --vacuum-size=200M
journalctl -r | grep "No space left on device"
docker system df
ordocker system df --verbose
docker system df -v | grep -A100 "VOLUME NAME"
docker system prune
ordocker system prune -a
ordocker system prune --volumes
docker buildx prune --keep-storage
find . -ls 2>/dev/null | sort -rnk7 | more
find -delete
du -hs -- * 2>/dev/null
du -hs /var/log/nagios3/ /var/log/sysstat/ /var/lib/prometheus/
ls -ltr | sort -rnk7
ls -ld $(find ./) | sort -rnk5 | more
growpart, resize2fs
sar -F --human
- macOS:
- overlay2
- Nagios: SERVICE ALERT: localhost;Disk Space;CRITICAL;SOFT;1;DISK CRITICAL
SystemMaxUse
of journalimjournal.state.tmp
Related software errors[edit]
- Artifactory:
2020-10-05T08:37:25.312Z [jfrt ] [WARN ] [887efcf06c050966] [r.s.RepositoryServiceImpl:1822] [tp-nio-8081-exec-289] - 90% storage space reached. Your maximum limit is 93%. It is recommended to add disk space.
- Argument list too long
Directories, filesystems[edit]
/var/log/
/tmp/
/proc/kcore
/var/lib/snapd/cache/
/var/cache/apt/archives/
/var/lib/apt/
/var/lib/snapd/snaps/
/var/lib/apt/lists/
/var/lib/prometheus/
/swap.img
/dev/sda1
kubectl edit pvc
pip cache purge
macOS[edit]
du -hs /Users/
du -hs /Users/*/brew/
du -hs /Users/*/Library/Containers/com.docker.docker 2>/dev/null
- ~/Library/Containers/com.docker.docker/Data/vms/0/data/Docker.raw
- GrandPerspective
Related[edit]
- Deallocate node
- Amazon Elastic Block Store (EBS): https://aws.amazon.com/premiumsupport/knowledge-center/ebs-volume-size-increase/
- MariaDB:
2023-03-30 5:06:42 4395974 [ERROR] mysqld: Disk full (/tmp/#sql_1b3c_0.MAI); waiting for someone to free some space... (errno: 28 "No space left on device")
See also[edit]
- File system,
du, df, find, ls, mkdir, touch, locate
- systemd-journald:
journalctl
,/etc/systemd/journald.conf
,journalctl logs
,journalctl --list-boots
,journalctl --disk-usage
,journalctl -u kubelet
,journalctl -u prometheus
,journalctl --help
- Monitoring: On call, Monitoring software, Monitoring services, Resource monitoring, Metric colletion tools, network monitoring, SLA Management Monitoring Tools, Alarm/Alert, Resource starvation, Alerts and notifications, Monitoring Kubernetes, VictoriaMetrics, Sensu, LogicMonitor, Distributed tracing, Datadog Monitors
du, du -hs
, Disk space usage,df
,ls
,tree
,find
,docker system df
,journalctl --disk-usage
, No space left on device, ENOSPC,sar -F
,growpart
,resize2fs
, ncdu, duf, dua-cli
Advertising: