Tech Kaizen: 9/1/19

Vector Packet Processing (VPP) introduction

The Vector Packet Processing(VPP) platform is an extensible framework that provides out-of-the-box production quality switch/router functionality.It is the open source version of Cisco's Vector Packet Processing (VPP) technology(a high performance packet-processing stack that can run on commodity CPUs). The benefits of this implementation of VPP are its high performance, proven technology, its modularity and flexibility, and rich feature set.

VPP is open source high-performance packet processing software. It leverages the Data Plane Development Kit (DPDK) to take advantage of fast I/O. DPDK provides fast packet processing libraries and user space drivers. It receives and send packets with a minimum number of CPU cycles by bypassing the kernel and using a user poll mode driver. Details on how to configure DPDK can be found in the DPDK Documentation. One of the benefits of this implementation of VPP is its high performance on relatively low-power computing.

This high level of performance is based on the following highlights:

High-performance user-space network stack for commodity hardware
The same code for host, VMs, Linux containers
Integrated vhost-user virtio backend for high speed VM-to-VM connectivity
L2 and L3 functionality, multiple encapsulations
Leverages best-of-breed open source driver technology: DPDK
Extensible by use of plugins
Control-plane / orchestration-plane via standards-based APIs

ref:

Vector Packet Processing(VPP) -

Data Plane Development Kit (DPDK) overview -

Build a Fast Network Stack with Vector Packet Processing (VPP) on an Intel® Architecture Server - https://software.intel.com/en-us/articles/build-a-fast-network-stack-with-vpp-on-an-intel-architecture-server

VPP vs DPDK - https://www.dpdk.org/wp-content/uploads/sites/35/2016/08/Day02-Session04-ThomasHerbert-DPDKUSASummit2016.pdf

PVPP(Programmable Vector Packet Processing) - https://web.stanford.edu/~yo2seol/static/talks/pvpp-cisco.pdf

Docker vs Kubernetes(K8s)

Containerization is a way of sharing parts of a single operating system among multiple isolated applications, as opposed to virtualization which will support multiple apps with their own OS on top of a single hypervisor. This means that where it can take minutes to spin up a virtual machine, it can take seconds to start a container because you aren't having to fire up the OS as well. This is beneficial for massive distributed applications with lots of discrete parts that need to be summoned, run, and then killed in short order.

The interest in containers led to the formation of the OpenContainer Initiative (OCI) to define the standards of container runtime and image formats. The industry is also witnessing various implementations of containers, such as LXD by Canonical, rkt by CoreOS, Windows Containers by Microsoft, CRI-O by KubernetesIncubator project, and vSphereIntegrated Containers by VMware.

Docker is an open platform for developing, shipping, and running applications. Docker enables you to separate your applications from your infrastructure so you can deliver software quickly. With Docker, you can manage your infrastructure in the same ways you manage your applications. Docker provides the ability to package and run an application in a loosely isolated environment called a container. The isolation and security allow you to run many containers simultaneously on a given host. A Docker container is a standard unit of software that packages up code and all its dependencies so the application runs quickly and reliably from one computing environment to another.

Kubernetes (K8s) is an open-source system for automating deployment, scaling, and management of containerized applications. It’s a container management platform designed to run enterprise-class, cloud-enabled and web-scalable IT workloads. It’s a Container Orchestration tool used mainly for provisioning containerized applications. Containers are microservices packaged with their dependencies and configurations. Kubernetes alternatives are Docker Swarm, Apache Mesos, OpenShift.

Docker vs Kubernetes:

“Kubernetes vs Docker” is like comparing apples to oranges. One isn’t an alternative to the other, they are complementary to each other. Quite the contrary; Kubernetes can run without Docker and Docker can function without Kubernetes.

Docker is a standalone software that can be installed on any computer to run containerized applications. Containerization is an approach of running applications on an OS such that the application is isolated from the rest of the system. You create an illusion for your application that it is getting its very own OS instance(but shared same kernel as the host), although there may be other containers running on same system. Docker is what enables us to run, create and manage containers on a single operating system.

If you have Docker installed on a bunch of hosts (different operating systems), you can leverage Kubernetes. These nodes, or Docker hosts, can be bare-metal servers or virtual machines. Kubernetes can then allow you to automate container provisioning, networking, load-balancing, security and scaling across all these nodes from a single command line or dashboard. A collection of nodes that is managed by a single Kubernetes instance is referred to as a Kubernetes cluster.

Go is the programming language of the Cloud. A large part of tools for cloud (Docker, Kubernetes, Istio) written in Go language.

ref:

Docker official website- https://www.docker.com/

Docker Documentation - https://docs.docker.com/

Docker Overview -
1. https://docs.docker.com/engine/docker-overview/
2. https://www.docker.com/resources/what-container
3. https://access.redhat.com/documentation/en-us/red_hat_enterprise_linux_atomic_host/7/html/overview_of_containers_in_red_hat_systems/introduction_to_linux_containers
4. https://ericchiang.github.io/post/containers-from-scratch/

Kubernetes official website - https://kubernetes.io/

What is Kubernetes - https://kubernetes.io/docs/concepts/overview/what-is-kubernetes/

Kubernetes basics - https://kubernetes.io/docs/tutorials/kubernetes-basics/

Kubernetes concepts - https://kubernetes.io/docs/concepts/

Kubernetes overview -

Why we use Go to develop Containerum platform for Kubernetes - https://medium.com/containerum/why-we-use-go-to-develop-containerum-platform-for-kubernetes-3a33d5bdc5ec

Kubernetes vs. Docker: A Primer - https://containerjournal.com/topics/container-ecosystems/kubernetes-vs-docker-a-primer/

Go, the Programming Language of the Cloud - https://thenewstack.io/go-the-programming-language-of-the-cloud/

Cloud Metrics Monitoring - StatsD, Graphite, Grafana

StatsD is a set of tools that can be used to send, collect, and aggregate custom metrics from any application. Graphite is enterprise-ready open source monitoring tool for storing and viewing time series data on any Cloud infrastructure. Grafana is an open source metric analytics & visualization suite. It is most commonly used for visualizing time series data for infrastructure and application analytics but many use it in other domains including industrial sensors, home automation, weather, and process control.

The most common setup is to use statsd to collect metrics, graphite to store those metrics, and Grafana to visualize those metrics(StatsD -> Graphite -> Grafana).

Graphite does not collect data by itself, but has a simple interface and integrates easily with third-party tools. Grafana allows you to connect to a Graphite installation (or other data source) and build dashboards to view and analyze the data.

StatsD client (e.g. golang client library) to collect and send the statistics and aggregation data to StatsD server. stats data are aggregated and sent to Graphite server. The metric data would be stored into Graphite server (include a Whisper database). Grafana is a powerful dashboard for visualizing the metrics data stored in Graphite(whisper) time-series database.

Prometheus is an open source monitoring and alerting toolkit. Prometheus is now closely integrated into cloud-native ecosystem and has native support for containers and Kubernetes.

cAdvisor (Container Advisor) provides container users an understanding of the resource usage and performance characteristics of their running containers(examples docker containers). It is a running daemon that collects, aggregates, processes, and exports information about running containers.

A time series database (TSDB) is a database optimized for time-stamped or time series data. Time series data are simply measurements or events that are tracked, monitored, sampled and aggregated over time. Applications like Self-driving cars, Autonomous trading, Smart homes monitoring sensor data rely on a form of data that measures how things change over time. Where time isn’t just a metric, but a primary axis. This is time-series data and it’s starting to play a larger role in our world.

ref:

StatsD, Graphite, Grafana -

Docker image with StatsD, Graphite, Grafana 2 - https://github.com/kamon-io/docker-grafana-graphite

Monitoring your Kubernetes Deployments with Prometheus - https://supergiant.io/blog/monitoring-your-kubernetes-deployments-with-prometheus/

Prometheus vs Grafana –

Grafana vs Kibana vs Prometheus - https://stackshare.io/stackups/grafana-vs-kibana-vs-prometheus