Difference between revisions of "Kubectl get events"

From wikieduonline
Jump to navigation Jump to search
 
(90 intermediate revisions by 3 users not shown)
Line 1: Line 1:
 
{{lc}}
 
{{lc}}
 +
TOMERGE: [[Kubernetes node events]]
 +
 +
* <code>[[kubectl get events --help]]</code>
 +
* <code>[[kubectl get events -A]]</code>
 +
* <code>[[kubectl get events -A]] | grep [[Warning]]</code>
 +
* <code>[[kubectl get events -A -o wide]] | grep [[Warning]]</code>
 
* <code>[[kubectl get]] events</code>
 
* <code>[[kubectl get]] events</code>
−
* <code>[[kubectl get events -A]]</code>
+
* <code>[[kubectl get events -o wide]]</code>
−
* <code>[[kubectl get event -A]]</code>
+
* <code>kubectl get events -o yaml</code>
−
* <code>[[kubectl -n gitlab-runner get events --field-selector type!=Normal]]</code>
+
 
 
* <code>[[kubectl get]] events --sort-by=.metadata.creationTimestamp</code>
 
* <code>[[kubectl get]] events --sort-by=.metadata.creationTimestamp</code>
 
* <code>[[kubectl get]] events --sort-by=.metadata.creationTimestamp -A</code>
 
* <code>[[kubectl get]] events --sort-by=.metadata.creationTimestamp -A</code>
 +
* <code>[[kubectl get]] events --sort-by='.lastTimestamp</code>
  
 +
* <code>[[kubectl -n gitlab-runner get events --field-selector type!=Normal]]</code>
 +
 +
 +
kubectl get events -A | grep [[Warning]] | egrep "[[FailedMount]]|[[FailedAttachVolume]]|[[Unhealthy]]|[[ClusterUnhealthy]]|[[FailedScheduling]]"
 +
 +
[[kubectl get events -A]] | grep [[Normal]]
  
 
== Events examples ==
 
== Events examples ==
 +
* [[Normal]], [[Warning]], [[Critical]]
 +
 
=== Warning ===
 
=== Warning ===
 +
 +
[[BackoffLimitExceeded]]
 +
[[CalculateExpectedPodCountFailed]]
 +
[[ClusterUnhealthy]]
 +
[[FailedMount]]
 +
[[FailedScheduling]]
 +
[[InvalidDiskCapacity]]
 +
[[Unhealthy]]
 +
.../...
 +
 +
 +
 +
your_namespace        28s        Warning  [[FailedScheduling]]                  pod/kibana-kibana-654ccb45bd-pbp4r            0/2 nodes are available: 2 [[Insufficient cpu]].
 +
 +
your_namespace        4m53s      Warning  [[ProbeWarning]]              [[pod]]/[[metabase]]-prod-f8f4b765b-h4pgs                                  Readiness probe warning:
 +
 +
your_namespace        30m        Warning  [[BackoffLimitExceeded]]      [[job/]]your-job27740460                          [[Job has reached the specified backoff limit]]
 +
 +
your_namespace        26m        Warning  [[Unhealthy]]                pod/elasticsearch-master-1                                          [[Readiness probe failed]]: Waiting for elasticsearch cluster to become ready (request params: "wait_for_status=green&timeout=1s" )...
 +
 
  your_namespace        99s        Warning  [[BackOff]]                pod/elasticsearch-master-0                Back-off restarting failed container
 
  your_namespace        99s        Warning  [[BackOff]]                pod/elasticsearch-master-0                Back-off restarting failed container
 
  your_namespace        108s        Warning  [[BackOff]]                pod/elasticsearch-master-1                [[Back-off restarting failed container]]
 
  your_namespace        108s        Warning  [[BackOff]]                pod/elasticsearch-master-1                [[Back-off restarting failed container]]
Line 15: Line 50:
 
  your_namespace        12m        Warning  [[PresentError]]          [[challenge]]/prod-admin-tls-cert-dzmbt-2545              [[Error presenting challenge]]: error getting [[clouddns]] service account: secret "clouddns-dns01-solver-svc-acct" [[secret not found|not found]]
 
  your_namespace        12m        Warning  [[PresentError]]          [[challenge]]/prod-admin-tls-cert-dzmbt-2545              [[Error presenting challenge]]: error getting [[clouddns]] service account: secret "clouddns-dns01-solver-svc-acct" [[secret not found|not found]]
  
−
  27m        Warning  [[OOMKilling]]        node/gke-you-pool4  Memory [[cgroup]] out of memory: Killed process 2768158 (python) total-vm:5613088kB, anon-rss:3051580kB, file-rss:65400kB, shmem-rss:0kB, UID:0 pgtables:7028kB oom_score_adj:997
+
  your_namespace 27m        Warning  [[OOMKilling]]        node/gke-you-pool4  Memory [[cgroup]] out of memory: Killed process 2768158 (python) total-vm:5613088kB, anon-rss:3051580kB, file-rss:65400kB, shmem-rss:0kB, UID:0 pgtables:7028kB oom_score_adj:997
  
−
  8m51s      Warning  [[FailedScheduling]]      pod/myprometheus-alertmanager-5967d4ff85-5glkh    running PreBind plugin "[[VolumeBinding]]": binding volumes: [[timed out waiting for the condition]]
+
  your_namespace 8m51s      Warning  [[FailedScheduling]]      pod/myprometheus-alertmanager-5967d4ff85-5glkh    running PreBind plugin "[[VolumeBinding]]": binding volumes: [[timed out waiting for the condition]]
 
  default    4m58s      Normal    [[ExternalProvisioning]]  persistentvolumeclaim/myprometheus-alertmanager  waiting for a volume to be created, either by external provisioner "[[ebs.csi.aws.com]]" or manually created by system administrator
 
  default    4m58s      Normal    [[ExternalProvisioning]]  persistentvolumeclaim/myprometheus-alertmanager  waiting for a volume to be created, either by external provisioner "[[ebs.csi.aws.com]]" or manually created by system administrator
 
   
 
   
Line 23: Line 58:
  
 
  default      107s        Warning  [[ProvisioningFailed]]    persistentvolumeclaim/myprometheus-server        (combined from similar events): failed to provision volume with StorageClass "[[gp2]]": rpc error: code = Internal desc = [[Could not create volume]] "pvc-4e14416c-c9c2-4d39-b749-9ce0fa98d597": could not create volume in EC2: [[UnauthorizedOperation]]: [[You are not authorized to perform this operation]]. Encoded authorization failure message: Goz6E3qExxxxx.../...
 
  default      107s        Warning  [[ProvisioningFailed]]    persistentvolumeclaim/myprometheus-server        (combined from similar events): failed to provision volume with StorageClass "[[gp2]]": rpc error: code = Internal desc = [[Could not create volume]] "pvc-4e14416c-c9c2-4d39-b749-9ce0fa98d597": could not create volume in EC2: [[UnauthorizedOperation]]: [[You are not authorized to perform this operation]]. Encoded authorization failure message: Goz6E3qExxxxx.../...
 +
 +
[[kube-system]]    9m44s      Warning  [[FailedMount]]              pod/[[kube-dns]]-85df8994db-v8qdg                          [[MountVolume]].SetUp failed for volume "kube-dns-config" : failed to sync [[configmap cache]]: [[timed out waiting for the condition]]</code>
 +
 +
 +
 
 +
[[kube-system]]    43m        [[Warning]]  [[ClusterUnhealthy]]          [[configmap/]]
 +
[[cluster-autoscaler-status]]   
 +
 +
     
 +
LAST SEEN  TYPE      REASON              OBJECT                                MESSAGE
 +
28s        Warning  FailedScheduling    pod/deployment-1230/3 nodes
 +
are available: 3 [[persistentvolumeclaim]] "your" bound to non-existent
 +
persistentvolume "".
 +
19m        Warning  FailedScheduling    pod/deployment-123      0/3 nodes
 +
are available: 3 persistentvolumeclaim "your" not found.
 +
10m        Warning  FailedScheduling    pod/deployment-91234      0/3 nodes
 +
are available: 3 persistentvolumeclaim "your" bound to non-existent
 +
persistentvolume "".
 +
17m        Warning  [[ClaimLost]]          persistentvolumeclaim/yourclaim  [[Bound claim has lost reference to PersistentVolume]]. Data on the volume is lost!          [[Cluster has no ready nodes]].
  
 
=== Normal ===
 
=== Normal ===
 +
Started
 +
Created
 +
Pulled
 +
[[Pulling]]
 +
[[Scheduled]]
 +
[[Killing]]
 +
[[Evict]]
 +
[[SandboxChanged]]
 +
[[SuccessfulCreate]] - [[ReplicaSet]]
 +
[[SuccessfulDelete]]
 +
[[NodeNotSchedulable]]
 +
[[RemovingNode]]
 +
[[TaintManagerEviction]]
 +
[[WaitForFirstConsumer]]
 +
[[ExternalProvisioning]]
 +
[[TaintManagerEviction: Cancelling deletion of pod]]
 +
 +
 +
 
  default    4s          Normal    [[Provisioning]]          persistentvolumeclaim/myprometheus-alertmanager  External provisioner is provisioning volume for claim "default/myprometheus-alertmanager"
 
  default    4s          Normal    [[Provisioning]]          persistentvolumeclaim/myprometheus-alertmanager  External provisioner is provisioning volume for claim "default/myprometheus-alertmanager"
 
   
 
   
 
  Related: <code>[[kubectl get pvc]]</code>
 
  Related: <code>[[kubectl get pvc]]</code>
 +
 +
 +
ingress-nginx  53m        Normal    [[UpdatedLoadBalancer]]      service/nginx-ingress-controller                        Updated load balancer with new hosts
 +
ingress-nginx  54m        Warning  [[UnAvailableLoadBalancer]]  service/nginx-ingress-controller                        There are no available nodes for LoadBalancer
  
 
== Events ==
 
== Events ==
−
[[BackOff]]
+
* <code>[[BackOff]]</code>
−
[[Completed]]
+
* <code>[[Completed]]</code>
−
[[Created]]
+
* <code>[[Created]]</code>
−
[[FailedMount]]
+
* <code>[[DeadlineExceeded]]</code>
−
[[FailedScheduling]]
+
* <code>[[Failed]]</code>
−
[[Generated]]
+
* <code>[[FailedAttachVolume]]</code>
−
[[PresentError]]
+
* <code>[[FailedCreatePodSandBox]]</code>
−
[[Pulled]]
+
* <code>[[FailedMount]]</code>
−
[[Pulling]]
+
* <code>[[FailedKillPod]]</code>
−
[[Requested]]
+
* <code>[[FailedScheduling]]</code>
−
[[SawCompletedJob]]
+
* <code>[[FailedToUpdateEndpoint]]</code>
−
[[Scheduled]]
+
* <code>[[FailedToUpdateEndpointSlices]]</code>
−
[[Started]]
+
* <code>[[Generated]]</code>
−
[[SuccessfulCreate]]
+
* <code>[[PresentError]]</code>
−
[[SuccessfulDelete]]
+
* <code>[[Pulled]]</code>
−
 
+
* <code>[[Pulling]]</code>
−
[[FailedKillPod]]
+
* <code>[[Requested]]</code>
 +
* <code>[[SawCompletedJob]]</code>
 +
* <code>[[Scheduled]]</code>
 +
* <code>[[Started]]</code>
 +
* <code>[[SuccessfulCreate]]</code>
 +
* <code>[[SuccessfulDelete]]</code>
 +
* <code>[[NetworkNotReady]]</code>
 +
* <code>[[NodeNotReady]]</code>
 +
* <code>[[NodeAllocatableEnforced]]</code>
 +
* <code>[[NoPods]]</code>
 +
* <code>[[NodeHasNoDiskPressure]]</code>
 +
* <code>[[UnAvailableLoadBalancer]]</code>
 +
* <code>[[Unhealthy]]</code>
 +
* <code>[[VolumeFailedDelete]]</code>
  
 
== Related ==
 
== Related ==
 +
* <code>[[kubectl events]]</code>
 
* <code>[[kubectl top]]</code>
 
* <code>[[kubectl top]]</code>
 
* <code>[[kubectl logs]]</code>
 
* <code>[[kubectl logs]]</code>
−
* <code>[[FailedScheduling]]</code>
+
* <code>[[gcloud logging read resource.labels.cluster_name]]</code>
−
* <code>[[kubectl get events --help]]</code>
 
 
* [[job-controller]]
 
* [[job-controller]]
−
* [[kubelet]]
 
 
* [[GCP Node logs]]
 
* [[GCP Node logs]]
−
* [[Events]]
+
* <code>[[gcloud logging read]]  projects/yourproject/logs/[[kubelet]]</code>
−
* [[gcloud logging read]]  projects/yourproject/logs/[[kubelet]]
+
* <code>[[kubectl describe nodes (conditions:)]]</code>
 +
* <code>[[kubectl describe nodes]] | grep [[KubeletReady]]</code>
 +
* <code>[[--event-ttl]]</code> defines the amount of time to retain events, default 1h.
  
 
== See also ==
 
== See also ==
 
* {{kubectl events}}
 
* {{kubectl events}}
−
* {{kubectl info}}
 
 
* {{K8s troubleshooting}}
 
* {{K8s troubleshooting}}
 +
* {{K8s monitoring}}
  
 
[[Category:K8s]]
 
[[Category:K8s]]

Latest revision as of 17:53, 10 April 2024

TOMERGE: Kubernetes node events 


kubectl get events -A | grep Warning | egrep "FailedMount|FailedAttachVolume|Unhealthy|ClusterUnhealthy|FailedScheduling"

kubectl get events -A | grep Normal

Events examples[edit]

Warning[edit]

BackoffLimitExceeded
CalculateExpectedPodCountFailed
ClusterUnhealthy 
FailedMount
FailedScheduling
InvalidDiskCapacity 
Unhealthy
.../...



your_namespace        28s         Warning   FailedScheduling                  pod/kibana-kibana-654ccb45bd-pbp4r             0/2 nodes are available: 2 Insufficient cpu.

your_namespace         4m53s       Warning   ProbeWarning              pod/metabase-prod-f8f4b765b-h4pgs                                   Readiness probe warning:

your_namespace         30m         Warning   BackoffLimitExceeded      job/your-job27740460                           Job has reached the specified backoff limit

your_namespace         26m         Warning   Unhealthy                 pod/elasticsearch-master-1                                          Readiness probe failed: Waiting for elasticsearch cluster to become ready (request params: "wait_for_status=green&timeout=1s" )...

your_namespace        99s         Warning   BackOff                pod/elasticsearch-master-0                Back-off restarting failed container
your_namespace        108s        Warning   BackOff                pod/elasticsearch-master-1                Back-off restarting failed container

your_namespace        12m         Warning   PresentError           challenge/prod-admin-tls-cert-dzmbt-2545              Error presenting challenge: error getting clouddns service account: secret "clouddns-dns01-solver-svc-acct" not found

your_namespace 27m         Warning   OOMKilling         node/gke-you-pool4   Memory cgroup out of memory: Killed process 2768158 (python) total-vm:5613088kB, anon-rss:3051580kB, file-rss:65400kB, shmem-rss:0kB, UID:0 pgtables:7028kB oom_score_adj:997

your_namespace 8m51s       Warning   FailedScheduling       pod/myprometheus-alertmanager-5967d4ff85-5glkh    running PreBind plugin "VolumeBinding": binding volumes: timed out waiting for the condition
default     4m58s       Normal    ExternalProvisioning   persistentvolumeclaim/myprometheus-alertmanager   waiting for a volume to be created, either by external provisioner "ebs.csi.aws.com" or manually created by system administrator

Solution: Install aws-ebs-csi-driver

default       107s        Warning   ProvisioningFailed     persistentvolumeclaim/myprometheus-server         (combined from similar events): failed to provision volume with StorageClass "gp2": rpc error: code = Internal desc = Could not create volume "pvc-4e14416c-c9c2-4d39-b749-9ce0fa98d597": could not create volume in EC2: UnauthorizedOperation: You are not authorized to perform this operation. Encoded authorization failure message: Goz6E3qExxxxx.../...

kube-system     9m44s       Warning   FailedMount               pod/kube-dns-85df8994db-v8qdg                           MountVolume.SetUp failed for volume "kube-dns-config" : failed to sync configmap cache: timed out waiting for the condition



kube-system     43m         Warning   ClusterUnhealthy          configmap/ 
cluster-autoscaler-status    



LAST SEEN   TYPE      REASON              OBJECT                                MESSAGE
28s         Warning   FailedScheduling    pod/deployment-1230/3 nodes 
are available: 3 persistentvolumeclaim "your" bound to non-existent 
persistentvolume "".
19m         Warning   FailedScheduling    pod/deployment-123       0/3 nodes 
are available: 3 persistentvolumeclaim "your" not found.
10m         Warning   FailedScheduling    pod/deployment-91234      0/3 nodes 
are available: 3 persistentvolumeclaim "your" bound to non-existent 
persistentvolume "".
17m         Warning   ClaimLost           persistentvolumeclaim/yourclaim   Bound claim has lost reference to PersistentVolume. Data on the volume is lost!           Cluster has no ready nodes.

Normal[edit]

Started
Created
Pulled
Pulling
Scheduled
Killing
Evict
SandboxChanged
SuccessfulCreate - ReplicaSet
SuccessfulDelete
NodeNotSchedulable
RemovingNode
TaintManagerEviction
WaitForFirstConsumer 
ExternalProvisioning
TaintManagerEviction: Cancelling deletion of pod



default     4s          Normal    Provisioning           persistentvolumeclaim/myprometheus-alertmanager   External provisioner is provisioning volume for claim "default/myprometheus-alertmanager"

Related: kubectl get pvc



ingress-nginx   53m         Normal    UpdatedLoadBalancer       service/nginx-ingress-controller                        Updated load balancer with new hosts
ingress-nginx   54m         Warning   UnAvailableLoadBalancer   service/nginx-ingress-controller                        There are no available nodes for LoadBalancer

Events[edit]

Related[edit]

See also[edit]

Retrieved from "https://www.wikieduonline.com/index.php?title=Kubectl_get_events&oldid=313391"

Advertising: