{ "cells": [ { "cell_type": "markdown", "metadata": {}, "source": [ "# Discover Cluster (ipynb)\n", "\n", "![discover_cluster_1](./discover_cluster_1.png)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## Install Dependencies" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "Install the dependencies `ipywidgets` and `kubectl`. Skip accordingly if they had already been installed." ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [ "# install ipywidgets \n", "!pip3 install ipywidgets" ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [ "# install kubectl\n", "!gcloud components install kubectl --quiet" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## Imports and Configuration" ] }, { "cell_type": "code", "execution_count": null, "id": "21d78493", "metadata": {}, "outputs": [], "source": [ "import ipywidgets as widgets\n", "import json\n", "import os\n", "\n", "from IPython.display import HTML, display\n", "\n", "# extend width of widgets\n", "display(HTML(''''''))\n", "# extend width of cells\n", "display(HTML(\"\"))\n", "display(HTML(\"\"))" ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [ "# [OPTIONAL] authenticate using your service account\n", "!gcloud auth activate-service-account --key-file " ] }, { "cell_type": "markdown", "id": "32702146", "metadata": {}, "source": [ "## Define Environment Variables" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "**Specify the following information**\n", "| Fields | Description |\n", "| ----------- | ----------- |\n", "| `Source Project` | Project id of target project that contains the k8s cluster |\n", "| `Cluster Name` | Name of k8s cluster |\n", "| `Cluster Type` | Type of k8s cluster (i.e. Regional or Zonal) |" ] }, { "cell_type": "code", "execution_count": null, "id": "3fcb000e", "metadata": {}, "outputs": [], "source": [ "# create text boxes for user input\n", "src_project = widgets.Text(description = \"Source Project: \", disabled=False)\n", "cluster_name = widgets.Text(description = \"Cluster Name: \", disabled=False)\n", "cluster_type = widgets.Dropdown(options=['Regional', \"Zonal\"], value='Zonal', description=\"Cluster Type: \", disabled=False)\n", "\n", "display(src_project, cluster_name, cluster_type)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "If `Cluster Type` is `Regional`, specify the `Cluster Region` (e.g. `asia-southeast1`). \n", "Else, if `Cluster Type` is `Zonal`, specify the `Cluster Zone` (e.g. `asia-southeast1-b`)." ] }, { "cell_type": "code", "execution_count": null, "id": "94a90614", "metadata": {}, "outputs": [], "source": [ "# create text boxes for user input\n", "if cluster_type.value == 'Regional':\n", " cluster_region = widgets.Text(description = \"Cluster Region: \", disabled=False)\n", " display(cluster_region)\n", "elif cluster_type.value == 'Zonal':\n", " cluster_zone = widgets.Text(description = \"Cluster Zone: \", disabled=False)\n", " display(cluster_zone)" ] }, { "cell_type": "code", "execution_count": null, "id": "4f0efea9", "metadata": {}, "outputs": [], "source": [ "# store user input in environment variables for use in subsequent comamnds\n", "os.environ['SRC_PROJECT'] = src_project.value\n", "os.environ['CLUSTER_NAME'] = cluster_name.value\n", "\n", "if cluster_type.value == 'Regional':\n", " os.environ['CLUSTER_REGION'] = cluster_region.value\n", "elif cluster_type.value == 'Zonal':\n", " os.environ['CLUSTER_ZONE'] = cluster_zone.value" ] }, { "cell_type": "markdown", "id": "4541b6cb", "metadata": {}, "source": [ "## Connect to Cluster" ] }, { "cell_type": "code", "execution_count": null, "id": "54d9567c", "metadata": {}, "outputs": [], "source": [ "if cluster_type.value == 'Regional':\n", " !gcloud container clusters get-credentials $CLUSTER_NAME --region $CLUSTER_REGION --project $SRC_PROJECT\n", "elif cluster_type.value == 'Zonal':\n", " !gcloud container clusters get-credentials $CLUSTER_NAME --zone $CLUSTER_ZONE --project $SRC_PROJECT" ] }, { "cell_type": "markdown", "id": "a0f54357", "metadata": {}, "source": [ "## List All" ] }, { "cell_type": "markdown", "id": "b1f533d4", "metadata": {}, "source": [ "**Append `kubectl` commands below with `--namespace ` to filter resource by namespace or `--all-namespaces` to include all namespaces**" ] }, { "cell_type": "code", "execution_count": null, "id": "df478680", "metadata": {}, "outputs": [], "source": [ "!kubectl get all --all-namespaces" ] }, { "cell_type": "markdown", "id": "0440c3fc", "metadata": {}, "source": [ "## List & Describe Nodes" ] }, { "cell_type": "code", "execution_count": null, "id": "85d7687c", "metadata": { "scrolled": false }, "outputs": [], "source": [ "# list nodes\n", "!kubectl get nodes -o wide --all-namespaces\n", "# sample output\n", "# NAME STATUS VERSION OS-IMAGE CONTAINER-RUNTIME\n", "# gke-node-1 Ready v1.16.15-gke.6000 Container-Optimized OS from Google docker://19.3.1\n", "# gke-node-2 Ready v1.16.15-gke.6000 Container-Optimized OS from Google docker://19.3.1\n", "# gke-node-3 Ready v1.16.15-gke.6000 Container-Optimized OS from Google docker://19.3.1\n", "# gke-node-4 Ready v1.19.6-gke.600 Container-Optimized OS from Google containerd://1.4.1\n", "# gke-node-5 Ready v1.19.6-gke.600 Container-Optimized OS from Google containerd://1.4.1\n", "# gke-node-6 Ready v1.19.6-gke.600 Container-Optimized OS from Google containerd://1.4.1" ] }, { "cell_type": "markdown", "id": "20394c60", "metadata": {}, "source": [ "- Take special note of the container runtime (`CONTAINER-RUNTIME` field), which determines how the filesystem can be mounted for analysis" ] }, { "cell_type": "code", "execution_count": null, "id": "0d4eb0b6", "metadata": {}, "outputs": [], "source": [ "# describe specific node\n", "!kubectl describe node \n", "# abstract of sample output\n", "# ...\n", "# CreationTimestamp: Thu, 29 Jul 2021 05:39:00 +0000\n", "# ...\n", "# Non-terminated Pods: (9 in total)\n", "# Namespace Name CPU Requests CPU Limits Memory Requests Memory Limits Age\n", "# --------- ---- ------------ ---------- --------------- ------------- ---\n", "# default nginx-deployment-b4db97b7f-j9hmv 0 (0%) 0 (0%) 0 (0%) 0 (0%) 40m\n", "# kube-system event-exporter-gke-67986489c8-jx5wj 0 (0%) 0 (0%) 0 (0%) 0 (0%) 25h\n", "# kube-system fluentbit-gke-gwhdf 100m (10%) 0 (0%) 200Mi (7%) 500Mi (17%) 25h\n", "# kube-system gke-metrics-agent-vkn7p 3m (0%) 0 (0%) 50Mi (1%) 50Mi (1%) 25h\n", "# kube-system kube-dns-autoscaler-58cbd4f75c-zv24d 20m (2%) 0 (0%) 10Mi (0%) 0 (0%) 25h\n", "# kube-system kube-proxy-gke-cluster-1-default-pool-fa6804d8-5x10 100m (10%) 0 (0%) 0 (0%) 0 (0%) 25h\n", "# kube-system metrics-server-v0.3.6-6c47ffd7d7-cfbwh 48m (5%) 143m (15%) 105Mi (3%) 355Mi (12%) 25h\n", "# kube-system pdcsi-node-t2pww 0 (0%) 0 (0%) 20Mi (0%) 100Mi (3%) 25h\n", "# kube-system stackdriver-metadata-agent-cluster-level-58855749c5-s9bbz 98m (10%) 48m (5%) 202Mi (7%) 202Mi (7%) 25h\n", "# ..." ] }, { "cell_type": "markdown", "id": "b0c4c1ea", "metadata": {}, "source": [ "- Take special note of `CreationTimestamp` and `Non-terminated Pods` in **non** `kube-system` namespace" ] }, { "cell_type": "markdown", "id": "721f6e48", "metadata": {}, "source": [ "## List & Describe Deployments" ] }, { "cell_type": "code", "execution_count": null, "id": "a155321a", "metadata": {}, "outputs": [], "source": [ "# list deployments\n", "!kubectl get deployments -o wide --all-namespaces\n", "# sample output\n", "# NAME READY UP-TO-DATE AVAILABLE AGE CONTAINERS IMAGES SELECTOR\n", "# nginx-deployment 3/3 3 3 5m2s web-container nginx app=nginx" ] }, { "cell_type": "code", "execution_count": null, "id": "80f80016", "metadata": {}, "outputs": [], "source": [ "# describe specific deployment\n", "!kubectl describe deployment --namespace \n", "# abstract of sample output\n", "# Name: nginx-deployment\n", "# Namespace: default\n", "# CreationTimestamp: Fri, 30 Jul 2021 06:44:29 +0000\n", "# Labels: app=nginx\n", "# Annotations: deployment.kubernetes.io/revision: 1\n", "# Selector: app=nginx\n", "# Replicas: 3 desired | 3 updated | 3 total | 3 available | 0 unavailable\n", "# StrategyType: RollingUpdate\n", "# MinReadySeconds: 0\n", "# RollingUpdateStrategy: 25% max unavailable, 25% max surge\n", "# Pod Template:\n", "# Labels: app=nginx\n", "# Containers:\n", "# web-container:\n", "# Image: nginx\n", "# Port: 80/TCP\n", "# Host Port: 0/TCP\n", "# Environment: \n", "# Mounts:\n", "# /usr/share/nginx/html from test-volume (rw)\n", "# Volumes:\n", "# test-volume:\n", "# Type: GCEPersistentDisk (a Persistent Disk resource in Google Compute Engine)\n", "# PDName: my-data-disk\n", "# FSType: ext4\n", "# Partition: 0\n", "# ReadOnly: true\n", "# ..." ] }, { "cell_type": "markdown", "id": "4e348aeb", "metadata": {}, "source": [ "- Take special note of `CreationTimestamp`, `Namespace`, `Labels`, `Replicas`, `Pod Template`" ] }, { "cell_type": "markdown", "id": "320aa5ea", "metadata": {}, "source": [ "## List & Describe Pods" ] }, { "cell_type": "code", "execution_count": null, "id": "d3475045", "metadata": {}, "outputs": [], "source": [ "# list pods\n", "!kubectl get pods -o wide --show-labels --all-namespaces\n", "# sample output\n", "# NAMESPACE NAME READY STATUS RESTARTS AGE IP NODE NOMINATED NODE READINESS GATES LABELS\n", "# default nginx-deployment-c9445c769-b5hsz 1/1 Running 0 67m 10.112.1.3 gke-cluster-1-default-pool-255ca9c3-tl9b app=nginx,pod-template-hash=c9445c769\n", "# default nginx-deployment-c9445c769-j5tm2 1/1 Running 0 63m 10.112.0.4 gke-cluster-1-default-pool-255ca9c3-v3ab app=nginx,pod-template-hash=c9445c769\n", "# default nginx-deployment-c9445c769-vwb7t 1/1 Running 0 71m 10.112.3.3 gke-cluster-1-default-pool-255ca9c3-qsne app=nginx,pod-template-hash=c9445c769\n", "# ..." ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [ "# list pods with containers\n", "# column CONTAINERS shows comma separated container names in a pod\n", "!kubectl get pods -o --all-namespaces 'custom-columns=NAMESPACE:.metadata.namespace,POD:.metadata.name,CONTAINERS:.spec.containers[*].name'\n", "# sample output\n", "# NAMESPACE POD CONTAINERS\n", "# default wordpress-1-deployer-d1w8s deployer\n", "# default wordpress-1-mysql-0 mysql,mysqld-exporter\n", "# default wordpress-1-wordpress-0 wordpress,apache-exporter" ] }, { "cell_type": "code", "execution_count": null, "id": "405de6bb", "metadata": {}, "outputs": [], "source": [ "# describe specific node\n", "!kubectl describe pod --namespace \n", "# abstract of sample output\n", "# Name: nginx-deployment-b4db97b7f-fvz6s\n", "# Namespace: default\n", "# Priority: 0\n", "# Node: gke-cluster-1-default-pool-fa6804d8-hr1r/10.148.0.10\n", "# Start Time: Fri, 30 Jul 2021 06:44:32 +0000\n", "# Labels: app=nginx\n", "# pod-template-hash=b4db97b7f\n", "# Annotations: \n", "# Status: Running\n", "# IP: 10.112.1.4\n", "# IPs:\n", "# IP: 10.112.1.4\n", "# Controlled By: ReplicaSet/nginx-deployment-b4db97b7f\n", "# Containers:\n", "# web-container:\n", "# Container ID: containerd://5bac9a80175f26ec75c5df1bddb09b670eef2feae90f6991c855c2fca3e28578\n", "# Image: nginx\n", "# Image ID: docker.io/library/nginx@sha256:8f335768880da6baf72b70c701002b45f4932acae8d574dedfddaf967fc3ac90\n", "# Port: 80/TCP\n", "# Host Port: 0/TCP\n", "# State: Running\n", "# Started: Fri, 30 Jul 2021 06:44:42 +0000\n", "# Ready: True\n", "# Restart Count: 0\n", "# Environment: \n", "# /usr/share/nginx/html from test-volume (rw)\n", "# /var/run/secrets/kubernetes.io/serviceaccount from default-token-6484k (ro)\n", "# Conditions:\n", "# Type Status\n", "# Initialized True\n", "# Ready True\n", "# ContainersReady True\n", "# PodScheduled True\n", "# Volumes:\n", "# test-volume:\n", "# Type: GCEPersistentDisk (a Persistent Disk resource in Google Compute Engine)\n", "# PDName: my-data-disk\n", "# FSType: ext4\n", "# Partition: 0\n", "# ReadOnly: true\n", "# ..." ] }, { "cell_type": "markdown", "id": "71fe2640", "metadata": {}, "source": [ "- Take special note of `Namespace`, `Node`, `Start Time`, `Labels`, `Containers`, `Image Volumes`\n", "- `Image` could reveal the exact containerized application, where it was downloaded from and its default configuration" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## List & Describe Services" ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [ "# list services\n", "!kubectl get services --all-namespaces\n", "# sample output\n", "# NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE\n", "# kubernetes ClusterIP 10.64.16.1 443/TCP 7d19h\n", "# wordpress-1-apache-exporter-svc ClusterIP None 9117/TCP 7d19h\n", "# wordpress-1-mysql-svc ClusterIP None 3306/TCP 7d19h\n", "# wordpress-1-mysqld-exporter-svc ClusterIP None 9104/TCP 7d19h\n", "# wordpress-1-wordpress-svc LoadBalancer 10.64.29.229 34.129.29.205 80:32221/TCP 7d19h" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "- To surface services exposed to Internet, look for TYPE `LoadBalancer` and `External-IP` is a public IP address" ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [ "# describe specific service\n", "!kubectl describe service --namespace \n", "# abstract of sample output\n", "# Name: wordpress-1-wordpress-svc\n", "# Namespace: default\n", "# Labels: app.kubernetes.io/component=wordpress-webserver\n", "# app.kubernetes.io/name=wordpress-1\n", "# Annotations: cloud.google.com/neg: {\"ingress\":true}\n", "# Selector: app.kubernetes.io/component=wordpress-webserver,app.kubernetes.io/name=wordpress-1\n", "# Type: LoadBalancer\n", "# IP Family Policy: SingleStack\n", "# IP Families: IPv4\n", "# IP: 10.64.29.229\n", "# IPs: 10.64.29.229\n", "# LoadBalancer Ingress: 34.129.29.205\n", "# Port: 80/TCP\n", "# TargetPort: 80/TCP\n", "# NodePort: 32221/TCP\n", "# Endpoints: 10.56.0.6:80\n", "# Session Affinity: None\n", "# External Traffic Policy: Cluster\n", "# Events: " ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "Based on above sample output\n", "- `Labels: app.kubernetes.io/name=wordpress-1` - Kubernetes application name\n", "- `IPs: 10.64.29.229` - IP addresses of the containers associated with the service\n", "- `Endpoints: 10.56.0.6:80` - IP address and port of the pods associated with the service" ] } ], "metadata": { "interpreter": { "hash": "e774977668b7c0ae8309835a5187aa7fbf7669e7d0bb59755bc63e573643edcd" }, "kernelspec": { "display_name": "Python 3.7.9 64-bit", "language": "python", "name": "python3" }, "language_info": { "codemirror_mode": { "name": "ipython", "version": 3 }, "file_extension": ".py", "mimetype": "text/x-python", "name": "python", "nbconvert_exporter": "python", "pygments_lexer": "ipython3", "version": "3.7.9" }, "widgets": { "application/vnd.jupyter.widget-state+json": { "state": {}, "version_major": 2, "version_minor": 0 } } }, "nbformat": 4, "nbformat_minor": 4 }