Kubernetes (K8s) Overview

Kubernetes(K8s) is an open-source container-orchestration system for automating application deployment, scaling, and management. It's all about automation of operational tasks. It was originally designed by Google, and is now maintained by the Cloud Native Computing Foundation(CNCF).

Google was one of the early contributors to Linux container technology. Google's internal platform, Borg was the predecessor to Kubernetes and the lessons learned from developing Borg over the years became the primary influence behind much of the Kubernetes technology.  It helps you fully implement and rely on a container-based infrastructure in production environments.

K3s is a lightweight, easy to install Kubernetes distribution geared towards resource-constrained environments and low touch operations. Some use cases in which k3s really shines are edge, ARM, IoT, and CI. K3s is a fully CNCF (Cloud Native Computing Foundation) certified Kubernetes offering. This means that you can write your YAML to operate against a regular “full-fat” Kubernetes and they’ll also apply against a k3s cluster.

Kubernetes key usages:

    1. Orchestrate containers across multiple hosts

    2. Control and automate application deployments and updates.

    3. Make better use of hardware to maximize resources needed to run your enterprise apps.

    4. Mount and add storage to run stateful apps.

    5. Scale containerized applications and their resources on the fly.

Kubernetes jargon:

    Master: The machine that controls Kubernetes nodes. This is where all task assignments originate.

    Node: These machines perform the requested, assigned tasks. The Kubernetes master controls them.

    Pod: Basic execution unit in Kubernetes. A group of one or more containers deployed to a single node. All containers in a pod share an IP address, IPC, hostname, and other resources. Pods abstract network and storage away from the underlying container. This lets you move containers around the cluster more easily.

    Replication controller:  This controls how many identical copies of a pod should be running somewhere on the cluster.

    Service: This decouples work definitions from the pods. Kubernetes service proxies automatically get service requests to the right pod—no matter where it moves to in the cluster or even if it’s been replaced.

    Kubelet: This service runs on nodes and reads the container manifests and ensures the defined containers are started and running.

    kubectl: This is the command line configuration tool for Kubernetes.


ref:

Kubernetes official website - https://kubernetes.io/

github Cloud computing source code -

    1. K8s source code(written in gloang) - https://github.com/kubernetes/kubernetes

    2. K3s source code(written in gloang) - https://github.com/rancher/k3s 

    3. Docker source code(written in glang) - https://github.com/docker, https://github.com/docker/docker.github.io

    4. Cloud Native Computing Foundation (CNCF) projects source code - https://github.com/cncf

    5. Apache Kafka(written in java): Open-source stream-processing software platform source code - https://github.com/apache/kafka

    6. The Apache Kafka C/C++ library - https://github.com/edenhill/librdkafka

A tour of the kubernetes source code part one: from kubectl to api server - https://developer.ibm.com/opentech/2017/06/21/tour-kubernetes-source-code-part-one-kubectl-api-server/

Kubernetes community - https://kubernetes.io/community/

Kubernetes Getting stated - https://kubernetes.io/docs/setup/

Kubernetes concepts - https://kubernetes.io/docs/concepts/

Kubernetes overview - https://thenewstack.io/kubernetes-an-overview/

Kubernetes Client libraries - https://kubernetes.io/docs/reference/using-api/client-libraries/

Debugging network stalls on Kubernetes - https://github.blog/2019-11-21-debugging-network-stalls-on-kubernetes/

Kubernetes components -  

    1. https://kubernetes.io/docs/concepts/overview/components/ 

    2. https://medium.com/jorgeacetozi/kubernetes-master-components-etcd-api-server-controller-manager-and-scheduler-3a0179fc8186

Kubernetes Cluster Architecture - https://netapp-trident.readthedocs.io/en/stable-v19.01/dag/kubernetes/kubernetes_cluster_architecture_considerations.html

Kubernetes Infrastructure - https://docs.okd.io/latest/architecture/infrastructure_components/kubernetes_infrastructure.html

Kubernetes Networking - https://itnext.io/kubernetes-networking-behind-the-scenes-39a1ab1792bb

Introducing k3s: The Lightweight Kubernetes Distribution Built for the Edge  - https://rancher.com/blog/2019/2019-02-26-introducing-k3s-the-lightweight-kubernetes-distribution-built-for-the-edge/

Kubernetes internals -

    1. Kubernetes internals links - https://github.com/shubheksha/kubernetes-internals 

    2. Kubernetes in Action online book -  https://livebook.manning.com/book/kubernetes-in-action/chapter-11/

    3. https://dev.to/ishankhare07/kubernetes-internals-14nk

    4. https://people.cs.umu.se/~larsson/2019/01/kubernetes-internals-running-a-pod/

What's Kubernetes -

    1.  https://www.redhat.com/en/topics/containers/what-is-kubernetes

    2.  https://kubernetes.io/docs/concepts/overview/what-is-kubernetes/

Aaron Schlesinger Kubernetes client sample code - 

    1.  https://github.com/arschles 

    2. https://github.com/aarons-talks/2017-KubeCon-EU

Kubernetes Youtube Videos -

    1. Kubernetes Deconstructed - https://www.youtube.com/watch?v=90kZRyPcRZw

    2. Kubernetes Architecture Made Easy - https://www.youtube.com/watch?v=_3NUI5vasPk

    3. Kubernetes Design Principles: Understand the Why - https://www.youtube.com/watch?v=ZuIQurh_kDk

    4. Inside Kubernetes An Architectural Deep Dive - https://www.youtube.com/watch?v=d5Rmu3fboiQ&t=389s
 
    5. Kubernetes Storage SIG - https://www.youtube.com/watch?v=PcMairR7-4U
 

Research and Development Publications/WhitePapers

R&D Publications/WhitePapers:

    1. Scientific Papers/Publications search - https://www.researchgate.net/search

    2. Cornell University Papers/Publications - https://arxiv.org/

    3. Google Artifcial Intelligence(AI) Research Papers - https://ai.google/research/pubs/ 
 
    4. Google Scholar articles - https://scholar.google.com/ 

    5. American Statistical Association Publications -  https://stattrak.amstat.org/
 
    6. Massachusetts Institute of Technology(MIT) Research - https://www.media.mit.edu/research/?filter=publications

    7. Wharton University Research - https://knowledge.wharton.upenn.edu/category/research/

    8. Harward Business Review - https://hbr.org/

    9. Search programming code - https://searchcode.com/
   
    10. Startup stories - https://inc42.com/startups/ 

    11. USA Small Business Administration(SBA) - https://www.sba.gov/
 
    12. Artificial Intelligence(AI) news - https://www.nextplatform.com/category/analyze/

    13. Edge Computing news - https://www.nextplatform.com/category/edge/ 

    14. Stanford online courses - https://online.stanford.edu/programs
   
    15. TIME Magazine - https://time.com/

Python for Beginners

Python is an interpreted, high-level, general-purpose programming language. Created by Guido van Rossum and first released in 1991, Python's design philosophy emphasizes code readability with its notable use of significant whitespace.

Python is an easy to learn, powerful programming language. It has efficient high-level data structures and a simple but effective approach to object-oriented programming. Python’s elegant syntax and dynamic typing, together with its interpreted nature, make it an ideal language for scripting and rapid application development in many areas on most platforms.

Data Analytics and Machine Learning are the keys to the future and Python plays a significant role in this domain. Most of the Data analytics software including Pandas, NumPy, SciPy, StatModels, Bokeh, Scikit-learn are Python based. Python Deep learning packages include TensorFlow, Keras, Elephas. Anaconda is a free and open-source distribution of the Python and R programming languages for scientific computing (data science, machine learning applications, large-scale data processing, predictive analytics etc) that aims to simplify package management and deployment.

ref:

Python official website - https://www.python.org/

Python tutorial - https://docs.python.org/3/tutorial/

Learn Python - https://www.learnpython.org/  

Python Beginners Videos(Microsoft Developer Series) - https://www.youtube.com/playlist?list=PLlrxD0HtieHhS8VzuMCfQD4uJ9yne1mE6 

Python Beginners guide:

    1.  https://wiki.python.org/moin/BeginnersGuide

    2.  https://www.python.org/about/gettingstarted/

Python documentation:

    1. Python 3.x documentation - https://docs.python.org/3/

    2. Python 2.x documentation - https://docs.python.org/2/

Python Playground:

    1. Python 3 Playground - https://www.katacoda.com/courses/python/playground 

    2. Python 3 Playground - https://code.sololearn.com/cOAXyhEmN1f7#py

    3. Python 2 Playground - https://code.sololearn.com/cd6pPeiuunQK/#py 

    4. Python 2 Playground - https://repl.it/@sergiitk/Python-2-playground 

Best Python Books - https://realpython.com/best-python-books/

Python Books - https://hackr.io/blog/best-python-books-for-beginners-and-advanced-programmers

Python Crash Course - https://github.com/topics/python-crash-course

Python playground source code - https://github.com/berkerpeksag/python-playground 

Data Science:

    1. Python Data Science distribution, Anaconda - https://www.anaconda.com/
   
    2. Top Python Libraries in DataScience - https://towardsdatascience.com/top-python-libraries-used-in-data-science-a58e90f1b4ba

    3. Python DataScience Handbook - https://jakevdp.github.io/PythonDataScienceHandbook/ 

    4. Python Data Science Tutorials - https://realpython.com/tutorials/data-science/

    5. A Complete Python Tutorial to Learn Data Science from Scratch - https://www.analyticsvidhya.com/blog/2016/01/complete-tutorial-learn-data-science-python-scratch-2

    6. Python Data Science introduction - https://cognitiveclass.ai/courses/python-for-data-science

    7. Learn Python for Data Science from scratch - https://data36.com/learn-python-for-data-science-from-scratch/

    8. Programming for Data Science with Python - https://www.udacity.com/course/programming-for-data-science-nanodegree--nd10

    9. github white paper on deep learning, genomics and precision medicine - https://github.com/greenelab/deep-review

Vector Packet Processing (VPP) introduction

The Vector Packet Processing(VPP) platform is an extensible framework that provides out-of-the-box production quality switch/router functionality.It is the open source version of Cisco's Vector Packet Processing (VPP) technology(a high performance packet-processing stack that can run on commodity CPUs). The benefits of this implementation of VPP are its high performance, proven technology, its modularity and flexibility, and rich feature set.

VPP is open source high-performance packet processing software. It leverages the Data Plane Development Kit (DPDK) to take advantage of fast I/O. DPDK provides fast packet processing libraries and user space drivers. It receives and send packets with a minimum number of CPU cycles by bypassing the kernel and using a user poll mode driver. Details on how to configure DPDK can be found in the DPDK Documentation. One of the benefits of this implementation of VPP is its high performance on relatively low-power computing. 

This high level of performance is based on the following highlights:

•  High-performance user-space network stack for commodity hardware
• The same code for host, VMs, Linux containers
• Integrated vhost-user virtio backend for high speed VM-to-VM connectivity
• L2 and L3 functionality, multiple encapsulations
• Leverages best-of-breed open source driver technology: DPDK
• Extensible by use of plugins
• Control-plane / orchestration-plane via standards-based APIs

ref:

Vector Packet Processing(VPP) -

1. https://fd.io/technology/
2. https://wiki.fd.io/view/VPP/What_is_VPP%3F
3. https://buildmedia.readthedocs.org/media/pdf/fdio-vpp/latest/fdio-vpp.pdf
4. https://object-storage-ca-ymq-1.vexxhost.net/swift/v1/6e4619c416ff4bd19e1c087f27a43eea/www-assets-prod/presentation-media/VPP-Barcelona.pdf

Data Plane Development Kit (DPDK) overview -

1. https://www.linuxjournal.com/content/userspace-networking-dpdk
2. https://pdfs.semanticscholar.org/e6d9/e0b89795005d7d5d4b0464bf7271ddd02ac0.pdf
3. https://doc.dpdk.org/guides/prog_guide/overview.html
4. https://blog.selectel.com/introduction-dpdk-architecture-principles/

Build a Fast Network Stack with Vector Packet Processing (VPP) on an Intel® Architecture Server - https://software.intel.com/en-us/articles/build-a-fast-network-stack-with-vpp-on-an-intel-architecture-server

VPP vs DPDK - https://www.dpdk.org/wp-content/uploads/sites/35/2016/08/Day02-Session04-ThomasHerbert-DPDKUSASummit2016.pdf

PVPP(Programmable Vector Packet Processing) - https://web.stanford.edu/~yo2seol/static/talks/pvpp-cisco.pdf

Docker vs Kubernetes(K8s)

Containerization is a way of sharing parts of a single operating system among multiple isolated applications, as opposed to virtualization which will support multiple apps with their own OS on top of a single hypervisor. This means that where it can take minutes to spin up a virtual machine, it can take seconds to start a container because you aren't having to fire up the OS as well. This is beneficial for massive distributed applications with lots of discrete parts that need to be summoned, run, and then killed in short order. 

The interest in containers led to the formation of the OpenContainer Initiative (OCI) to define the standards of container runtime and image formats. The industry is also witnessing various implementations of containers, such as LXD by Canonical, rkt by CoreOS, Windows Containers by Microsoft, CRI-O by KubernetesIncubator project, and vSphereIntegrated Containers by VMware.

Docker is an open platform for developing, shipping, and running applications. Docker enables you to separate your applications from your infrastructure so you can deliver software quickly. With Docker, you can manage your infrastructure in the same ways you manage your applications. Docker provides the ability to package and run an application in a loosely isolated environment called a container. The isolation and security allow you to run many containers simultaneously on a given host. A Docker container is a standard unit of software that packages up code and all its dependencies so the application runs quickly and reliably from one computing environment to another.
 
Kubernetes (K8s) is an open-source system for automating deployment, scaling, and management of containerized applications. It’s a container management platform designed to run enterprise-class, cloud-enabled and web-scalable IT workloads. It’s a Container Orchestration tool used mainly for provisioning containerized applications. Containers are microservices packaged with their dependencies and configurations. Kubernetes alternatives are Docker Swarm, Apache Mesos, OpenShift.

Docker vs Kubernetes: 

“Kubernetes vs Docker” is like comparing apples to oranges. One isn’t an alternative to the other, they are complementary to each other. Quite the contrary; Kubernetes can run without Docker and Docker can function without Kubernetes.

Docker is a standalone software that can be installed on any computer to run containerized applications. Containerization is an approach of running applications on an OS such that the application is isolated from the rest of the system. You create an illusion for your application that it is getting its very own OS instance(but shared same kernel as the host), although there may be other containers running on same system. Docker is what enables us to run, create and manage containers on a single operating system.

If you have Docker installed on a bunch of hosts (different operating systems), you can leverage Kubernetes. These nodes, or Docker hosts, can be bare-metal servers or virtual machines. Kubernetes can then allow you to automate container provisioning, networking, load-balancing, security and scaling across all these nodes from a single command line or dashboard. A collection of nodes that is managed by a single Kubernetes instance is referred to as a Kubernetes cluster.

Go is the programming language of the Cloud. A large part of tools for cloud (Docker, Kubernetes, Istio) written in Go language.

ref: 

Docker official website- https://www.docker.com/ 

Docker Documentation - https://docs.docker.com/ 

Docker Overview -
    1. https://docs.docker.com/engine/docker-overview/
    2. https://www.docker.com/resources/what-container
    3. https://access.redhat.com/documentation/en-us/red_hat_enterprise_linux_atomic_host/7/html/overview_of_containers_in_red_hat_systems/introduction_to_linux_containers
    4. https://ericchiang.github.io/post/containers-from-scratch/

Kubernetes official website - https://kubernetes.io/ 

What is Kubernetes - https://kubernetes.io/docs/concepts/overview/what-is-kubernetes/ 

Kubernetes basics - https://kubernetes.io/docs/tutorials/kubernetes-basics/ 

Kubernetes concepts - https://kubernetes.io/docs/concepts/

Kubernetes overview -

1. https://thenewstack.io/kubernetes-an-overview/
2. https://azure.microsoft.com/en-us/topic/what-is-kubernetes/?&OCID=AID2000586_SEM_uQFwC39m&lnkd=Google_Kubernetes_Nonbrand&MarinID=suQFwC39m_372584111946_%2Bkubernetes_b_c__75821059109_kwd-88228236663_
3. https://justanotherdevblog.com/2017/02/22/kubernetes-an-overview-bf47b0af1865/

Why we use Go to develop Containerum platform for Kubernetes - https://medium.com/containerum/why-we-use-go-to-develop-containerum-platform-for-kubernetes-3a33d5bdc5ec 

Kubernetes vs. Docker: A Primer - https://containerjournal.com/topics/container-ecosystems/kubernetes-vs-docker-a-primer/ 

Go, the Programming Language of the Cloud - https://thenewstack.io/go-the-programming-language-of-the-cloud/