Difference between revisions of "Kubectl get events"

From wikieduonline
Jump to navigation Jump to search
 
(58 intermediate revisions by 2 users not shown)
Line 1: Line 1:
 
{{lc}}
 
{{lc}}
 +
TOMERGE: [[Kubernetes node events]]
 +
 +
* <code>[[kubectl get events --help]]</code>
 +
* <code>[[kubectl get events -A]]</code>
 +
* <code>[[kubectl get events -A]] | grep [[Warning]]</code>
 +
* <code>[[kubectl get events -A -o wide]] | grep [[Warning]]</code>
 
* <code>[[kubectl get]] events</code>
 
* <code>[[kubectl get]] events</code>
−
* <code>[[kubectl get events -A]]</code>
+
* <code>[[kubectl get events -o wide]]</code>
 
* <code>kubectl get events -o yaml</code>
 
* <code>kubectl get events -o yaml</code>
  
Line 10: Line 16:
 
* <code>[[kubectl -n gitlab-runner get events --field-selector type!=Normal]]</code>
 
* <code>[[kubectl -n gitlab-runner get events --field-selector type!=Normal]]</code>
  
−
  kubectl get events -A | grep [[Warning]]
+
 
−
kubectl get events -A | grep [[Warning]] | egrep "[[FailedMount]]"
+
  kubectl get events -A | grep [[Warning]] | egrep "[[FailedMount]]|[[FailedAttachVolume]]|[[Unhealthy]]|[[ClusterUnhealthy]]|[[FailedScheduling]]"
 +
 
 
  [[kubectl get events -A]] | grep [[Normal]]
 
  [[kubectl get events -A]] | grep [[Normal]]
  
 
== Events examples ==
 
== Events examples ==
 +
* [[Normal]], [[Warning]], [[Critical]]
 +
 
=== Warning ===
 
=== Warning ===
 +
 +
[[BackoffLimitExceeded]]
 +
[[CalculateExpectedPodCountFailed]]
 +
[[ClusterUnhealthy]]
 +
[[FailedMount]]
 +
[[FailedScheduling]]
 +
[[InvalidDiskCapacity]]
 +
[[Unhealthy]]
 +
.../...
 +
 +
  
 
  your_namespace        28s        Warning  [[FailedScheduling]]                  pod/kibana-kibana-654ccb45bd-pbp4r            0/2 nodes are available: 2 [[Insufficient cpu]].
 
  your_namespace        28s        Warning  [[FailedScheduling]]                  pod/kibana-kibana-654ccb45bd-pbp4r            0/2 nodes are available: 2 [[Insufficient cpu]].
Line 43: Line 63:
  
 
    
 
    
−
  [[kube-system]]    43m        [[Warning]]  [[ClusterUnhealthy]]          [[configmap/]][[cluster-autoscaler-status]]                     [[Cluster has no ready nodes]].
+
  [[kube-system]]    43m        [[Warning]]  [[ClusterUnhealthy]]          [[configmap/]]  
 +
[[cluster-autoscaler-status]]  
 +
 
 +
     
 +
LAST SEEN  TYPE      REASON              OBJECT                                MESSAGE
 +
28s        Warning  FailedScheduling    pod/deployment-1230/3 nodes
 +
are available: 3 [[persistentvolumeclaim]] "your" bound to non-existent
 +
persistentvolume "".
 +
19m        Warning  FailedScheduling    pod/deployment-123      0/3 nodes
 +
are available: 3 persistentvolumeclaim "your" not found.
 +
10m        Warning  FailedScheduling    pod/deployment-91234      0/3 nodes
 +
are available: 3 persistentvolumeclaim "your" bound to non-existent
 +
persistentvolume "".
 +
17m        Warning  [[ClaimLost]]          persistentvolumeclaim/yourclaim  [[Bound claim has lost reference to PersistentVolume]]. Data on the volume is lost!          [[Cluster has no ready nodes]].
  
 
=== Normal ===
 
=== Normal ===
 +
Started
 +
Created
 +
Pulled
 +
[[Pulling]]
 +
[[Scheduled]]
 +
[[Killing]]
 +
[[Evict]]
 +
[[SandboxChanged]]
 +
[[SuccessfulCreate]] - [[ReplicaSet]]
 +
[[SuccessfulDelete]]
 +
[[NodeNotSchedulable]]
 +
[[RemovingNode]]
 +
[[TaintManagerEviction]]
 +
[[WaitForFirstConsumer]]
 +
[[ExternalProvisioning]]
 +
[[TaintManagerEviction: Cancelling deletion of pod]]
 +
 +
 +
 
  default    4s          Normal    [[Provisioning]]          persistentvolumeclaim/myprometheus-alertmanager  External provisioner is provisioning volume for claim "default/myprometheus-alertmanager"
 
  default    4s          Normal    [[Provisioning]]          persistentvolumeclaim/myprometheus-alertmanager  External provisioner is provisioning volume for claim "default/myprometheus-alertmanager"
 
   
 
   
 
  Related: <code>[[kubectl get pvc]]</code>
 
  Related: <code>[[kubectl get pvc]]</code>
 +
 +
 +
ingress-nginx  53m        Normal    [[UpdatedLoadBalancer]]      service/nginx-ingress-controller                        Updated load balancer with new hosts
 +
ingress-nginx  54m        Warning  [[UnAvailableLoadBalancer]]  service/nginx-ingress-controller                        There are no available nodes for LoadBalancer
  
 
== Events ==
 
== Events ==
Line 54: Line 110:
 
* <code>[[Completed]]</code>
 
* <code>[[Completed]]</code>
 
* <code>[[Created]]</code>
 
* <code>[[Created]]</code>
 +
* <code>[[DeadlineExceeded]]</code>
 +
* <code>[[Failed]]</code>
 +
* <code>[[FailedAttachVolume]]</code>
 +
* <code>[[FailedCreatePodSandBox]]</code>
 
* <code>[[FailedMount]]</code>
 
* <code>[[FailedMount]]</code>
 +
* <code>[[FailedKillPod]]</code>
 
* <code>[[FailedScheduling]]</code>
 
* <code>[[FailedScheduling]]</code>
 +
* <code>[[FailedToUpdateEndpoint]]</code>
 +
* <code>[[FailedToUpdateEndpointSlices]]</code>
 
* <code>[[Generated]]</code>
 
* <code>[[Generated]]</code>
 
* <code>[[PresentError]]</code>
 
* <code>[[PresentError]]</code>
Line 66: Line 129:
 
* <code>[[SuccessfulCreate]]</code>
 
* <code>[[SuccessfulCreate]]</code>
 
* <code>[[SuccessfulDelete]]</code>
 
* <code>[[SuccessfulDelete]]</code>
−
* <code>[[FailedKillPod]]</code>
+
* <code>[[NetworkNotReady]]</code>
 +
* <code>[[NodeNotReady]]</code>
 +
* <code>[[NodeAllocatableEnforced]]</code>
 
* <code>[[NoPods]]</code>
 
* <code>[[NoPods]]</code>
 
* <code>[[NodeHasNoDiskPressure]]</code>
 
* <code>[[NodeHasNoDiskPressure]]</code>
−
* [[FailedAttachVolume]]
+
* <code>[[UnAvailableLoadBalancer]]</code>
 +
* <code>[[Unhealthy]]</code>
 +
* <code>[[VolumeFailedDelete]]</code>
  
 
== Related ==
 
== Related ==
 +
* <code>[[kubectl events]]</code>
 
* <code>[[kubectl top]]</code>
 
* <code>[[kubectl top]]</code>
 
* <code>[[kubectl logs]]</code>
 
* <code>[[kubectl logs]]</code>
−
* <code>[[FailedScheduling]]</code>
+
* <code>[[gcloud logging read resource.labels.cluster_name]]</code>
−
* <code>[[kubectl get events --help]]</code>
 
 
* [[job-controller]]
 
* [[job-controller]]
−
* [[kubelet]]
 
 
* [[GCP Node logs]]
 
* [[GCP Node logs]]
−
* [[Events]]
 
 
* <code>[[gcloud logging read]]  projects/yourproject/logs/[[kubelet]]</code>
 
* <code>[[gcloud logging read]]  projects/yourproject/logs/[[kubelet]]</code>
 
* <code>[[kubectl describe nodes (conditions:)]]</code>
 
* <code>[[kubectl describe nodes (conditions:)]]</code>
 +
* <code>[[kubectl describe nodes]] | grep [[KubeletReady]]</code>
 +
* <code>[[--event-ttl]]</code> defines the amount of time to retain events, default 1h.
  
 
== See also ==
 
== See also ==
 
* {{kubectl events}}
 
* {{kubectl events}}
−
* {{kubectl info}}
 
 
* {{K8s troubleshooting}}
 
* {{K8s troubleshooting}}
 +
* {{K8s monitoring}}
  
 
[[Category:K8s]]
 
[[Category:K8s]]

Latest revision as of 17:53, 10 April 2024

TOMERGE: Kubernetes node events 


kubectl get events -A | grep Warning | egrep "FailedMount|FailedAttachVolume|Unhealthy|ClusterUnhealthy|FailedScheduling"

kubectl get events -A | grep Normal

Events examples[edit]

Warning[edit]

BackoffLimitExceeded
CalculateExpectedPodCountFailed
ClusterUnhealthy 
FailedMount
FailedScheduling
InvalidDiskCapacity 
Unhealthy
.../...



your_namespace        28s         Warning   FailedScheduling                  pod/kibana-kibana-654ccb45bd-pbp4r             0/2 nodes are available: 2 Insufficient cpu.

your_namespace         4m53s       Warning   ProbeWarning              pod/metabase-prod-f8f4b765b-h4pgs                                   Readiness probe warning:

your_namespace         30m         Warning   BackoffLimitExceeded      job/your-job27740460                           Job has reached the specified backoff limit

your_namespace         26m         Warning   Unhealthy                 pod/elasticsearch-master-1                                          Readiness probe failed: Waiting for elasticsearch cluster to become ready (request params: "wait_for_status=green&timeout=1s" )...

your_namespace        99s         Warning   BackOff                pod/elasticsearch-master-0                Back-off restarting failed container
your_namespace        108s        Warning   BackOff                pod/elasticsearch-master-1                Back-off restarting failed container

your_namespace        12m         Warning   PresentError           challenge/prod-admin-tls-cert-dzmbt-2545              Error presenting challenge: error getting clouddns service account: secret "clouddns-dns01-solver-svc-acct" not found

your_namespace 27m         Warning   OOMKilling         node/gke-you-pool4   Memory cgroup out of memory: Killed process 2768158 (python) total-vm:5613088kB, anon-rss:3051580kB, file-rss:65400kB, shmem-rss:0kB, UID:0 pgtables:7028kB oom_score_adj:997

your_namespace 8m51s       Warning   FailedScheduling       pod/myprometheus-alertmanager-5967d4ff85-5glkh    running PreBind plugin "VolumeBinding": binding volumes: timed out waiting for the condition
default     4m58s       Normal    ExternalProvisioning   persistentvolumeclaim/myprometheus-alertmanager   waiting for a volume to be created, either by external provisioner "ebs.csi.aws.com" or manually created by system administrator

Solution: Install aws-ebs-csi-driver

default       107s        Warning   ProvisioningFailed     persistentvolumeclaim/myprometheus-server         (combined from similar events): failed to provision volume with StorageClass "gp2": rpc error: code = Internal desc = Could not create volume "pvc-4e14416c-c9c2-4d39-b749-9ce0fa98d597": could not create volume in EC2: UnauthorizedOperation: You are not authorized to perform this operation. Encoded authorization failure message: Goz6E3qExxxxx.../...

kube-system     9m44s       Warning   FailedMount               pod/kube-dns-85df8994db-v8qdg                           MountVolume.SetUp failed for volume "kube-dns-config" : failed to sync configmap cache: timed out waiting for the condition



kube-system     43m         Warning   ClusterUnhealthy          configmap/ 
cluster-autoscaler-status    



LAST SEEN   TYPE      REASON              OBJECT                                MESSAGE
28s         Warning   FailedScheduling    pod/deployment-1230/3 nodes 
are available: 3 persistentvolumeclaim "your" bound to non-existent 
persistentvolume "".
19m         Warning   FailedScheduling    pod/deployment-123       0/3 nodes 
are available: 3 persistentvolumeclaim "your" not found.
10m         Warning   FailedScheduling    pod/deployment-91234      0/3 nodes 
are available: 3 persistentvolumeclaim "your" bound to non-existent 
persistentvolume "".
17m         Warning   ClaimLost           persistentvolumeclaim/yourclaim   Bound claim has lost reference to PersistentVolume. Data on the volume is lost!           Cluster has no ready nodes.

Normal[edit]

Started
Created
Pulled
Pulling
Scheduled
Killing
Evict
SandboxChanged
SuccessfulCreate - ReplicaSet
SuccessfulDelete
NodeNotSchedulable
RemovingNode
TaintManagerEviction
WaitForFirstConsumer 
ExternalProvisioning
TaintManagerEviction: Cancelling deletion of pod



default     4s          Normal    Provisioning           persistentvolumeclaim/myprometheus-alertmanager   External provisioner is provisioning volume for claim "default/myprometheus-alertmanager"

Related: kubectl get pvc



ingress-nginx   53m         Normal    UpdatedLoadBalancer       service/nginx-ingress-controller                        Updated load balancer with new hosts
ingress-nginx   54m         Warning   UnAvailableLoadBalancer   service/nginx-ingress-controller                        There are no available nodes for LoadBalancer

Events[edit]

Related[edit]

See also[edit]

Retrieved from "https://www.wikieduonline.com/index.php?title=Kubectl_get_events&oldid=313391"

Advertising: