amazon linux 2

Published 2019-05-05

Amazon Linux 2

Change hostname use init-cloud:

vim /etc/cloud/cloud.cfg.d/11_changehostname.cfg
#cloud-config
preserve_hostname: false
hostname: myhostname
fqdn: myhostname.example.com
manage_etc_hosts: true

By Command:

1 2	hostnamectl set-hostname foobar.localdomain sed -i '1s/localhost/foobar/1' /etc/hosts\|sed -i '1s/localhost/foobar/2' /etc/hosts

Extra Library:

1 2	[ec2-user ~]$ amazon-linux-extras list [ec2-user ~]$ sudo amazon-linux-extras install topic=version topic=version

docker

By Stan Zhou

Published 2019-05-05

what is docker

Open platform
Build, ship, and run applications
Popular because:
- Flexible
- Lightweight
- Interchangeable
- Portable
- Scalable
  List Docker CLI commands

docker
docker container --help

## Display Docker version and info
docker --version
docker version
docker info

## Execute Docker image
docker run hello-world

## List Docker images
docker image ls

## List Docker containers (running, all, all in quiet mode)
docker container ls
docker container ls --all
docker container ls -aq


docker build -t friendlyhello .  # Create image using this directory's Dockerfile
docker run -p 4000:80 friendlyhello  # Run "friendlyhello" mapping port 4000 to 80
docker run -d -p 4000:80 friendlyhello         # Same thing, but in detached mode
docker container ls                                # List all running containers
docker container ls -a             # List all containers, even those not running
docker container stop <hash>           # Gracefully stop the specified container
docker container kill <hash>         # Force shutdown of the specified container
docker container rm <hash>        # Remove specified container from this machine
docker container rm $(docker container ls -a -q)         # Remove all containers
docker image ls -a                             # List all images on this machine
docker image rm <image id>            # Remove specified image from this machine
docker image rm $(docker image ls -a -q)   # Remove all images from this machine
docker rm   # deletes a stopped container
docker rmi  # deletes a container image
docker login             # Log in this CLI session using your Docker credentials
docker tag <image> username/repository:tag  # Tag <image> for upload to registry
docker push username/repository:tag            # Upload tagged image to registry
docker run username/repository:tag                   # Run image from a registry
docker system prune -fa #will remove any container images that are not tied to an existing runningcontainer, along with any other resources in your Docker environment

Beginning Devops with Docker

Chapter 1. Images and Containers

Virtualization versus Containerization

In virtual machines, the physical hardware is abstracted, therefore we have may server running on one server. Virtual machines do sometimes take time to start up and are expensive in capacity (they can be GBs in size), although the greatest advantage they have over containers is the ability to run different Linux distributions such as CentOS instead of just Ubuntu.

In containerization, it is only the app layer (where code and dependencies are packaged) that is abstracted, making it possible for many containers to run on the same OS kernel but on separate user space.

Containers use less space and boot fast.

Describe how Docker improves a DevOps workflow
Normally, we have two pieces to a working application: the project code base and the provisioning script. The code base is the application code. It is managed by version control and hosted in GitHub, among other platforms.

The Dockerfile takes the place of the provisioning script. The two combined (project code and Dockerfile) make a Docker image. A Docker image can be run as an application. This running application sourced from a Docker image is called a Docker container.

Basic Docker Terminal Commands

docker info ## displays system-wide information
docker search <term> (foir example. docker search ubuntu) ##searches Docker Hub for Images
docker pull ## pulls an image from the registry to your local machine eg docker pull ubuntu
docker images ## lists the Docker images we have locally
docker run <image> #will check whether the image by the name hello-world exists locally,if the image is
not local, it will be pulled form the default registry, Docker Hub, and run as a container, by default

docker ps ## lists the running containers
docker stop <container ID>  ## stop the container 
docker rm <container ID> ## Delete the container and the image

Interpret Dockerfile syntax
Build images
Set up containers and images
Set up a local dynamic environment
Run applications in Docker containers
Obtain a basic overview of how Docker manages images via Docker Hub
Deploy a Docker image to Docker Hub

The Dockerfile

FROM openjdk:8-jre-alpine

ENV SPRING_OUTPUT_ANSI_ENABLED=ALWAYS \
    JHIPSTER_SLEEP=0 \
    JAVA_OPTS=""

CMD echo "The application will start in ${JHIPSTER_SLEEP}s..." && \
    sleep ${JHIPSTER_SLEEP} && \
    java ${JAVA_OPTS} -Djava.security.egd=file:/dev/./urandom -jar /app.war

EXPOSE 8080 5701/udp

ADD *.war /app.war

The FROM instruction specifies the base image to use while initializing the build. Here, we specify open JDK 8 as our Java runtime.

The ENV instruction is used to set environment variables, and the CMD instruction is used to specify commands to be executed.

The EXPOSE instruction is used to specify the port that the container listens to during runtime.

useful commands for Docker and Docker compose

Docker-compose

Sample docker-compose file that will start a Rdis database on port 5000

version: '3'
services:
  web:
    build: .
    ports:
     - "5000:5000"
  redis:
    image: "redis:alpine"

The first line of the docker-compose file should be the version of the docker-compose tool.

Then we need to specify all the necessary services that we need for our application to run. They should be defined in the services: section.

We can also define multiple services inside here, giving a name to each (web and redis).

This is followed by how to build the service (either via a command to build or referring a Dockerfile).

If the application needs any port access, we can configure it using 5000:5000 (that is internal port: external port).

Then, we have to specify the volume information. This basically tells docker-compose to serve the files from the location specified.

Once we have specified the services required for our application, then we can start the application via docker-compose. This will start your entire application along with the services, and expose the services on the port specified.

With docker-compose, we can perform the following operations:

Start: docker-compose -f up
Stop: docker-compose -f down
We can also perform the following operations:
List the running services and their status: docker ps
Logs: docker log

Kubernetes

A single deployable component is called a pod in Kubernetes. This can be as simple as a running process in the container. A group of pods can be combined together to form a deployment.

The following code is a sample Kubernetes file that will start a Nginx server:

apiVersion: v1
kind: Service
metadata:
  name: nginxsvc
  labels:
    app: nginx
spec:
  type: NodePort
  ports:
  - port: 80
    protocol: TCP
    name: http
  - port: 443
    protocol: TCP
    name: https
  selector:
    app: nginx

Start with an apiVersion in a Kubernetes deployment file.
Followed by the type, which takes either a pod, deployment, namespace, ingress (load balancing the pods), role, and many more.

Ingress forms a layer between the services and the internet so that all the inbound connections are controlled or configured with the ingress controller before sending them to Kubernetes services on the cluster. On the other hand, the egress controller controls or configures services going out of the Kubernetes cluster.

This is followed by the metadata information, such as the type of environments, the application name (nginxsvc), and labels (Kubernetes uses this information to identify and segregate the pods). Kubernetes uses this metadata information to identify the particular pods or a group of pods, and we can manage the instances with this metadata. This is one of the key differences with docker-compose, where docker-compose doesn’t have the flexibility of defining the metadata about the containers.
This is followed by the spec, where we define the specification of the images or our application. We can also define the pull strategy for our images as well as define the environment variables along with the exposed ports. We can define the resource limitations on the machine (or VM) for a particular service. They provide health checks, that is, each service is monitored for the health and when some services fail, they are immediately replaced by newer ones. They also provide service discovery out of the box, by assigning each pod an IP, which makes it easier for the services to identify and interact with them. They also provide a better dashboard, to visualize your architecture and the status of the application. You can do most of the management via this dashboard, such as checking the status, logs, scale up, or down the services, and so on.

Docker ps command

docker ps --format "table {{.ID}} \t {{.Image}} \t {{.Ports}} \t {{.Names}}"
CONTAINER ID         IMAGE                   PORTS                                                NAMES
7d3a6d518e8e         jenkinsci/blueocean     0.0.0.0:8080->8080/tcp, 0.0.0.0:50000->50000/tcp     jenkins
9655eb0e836e         alpine                                                                       quotes
root@stan-OptiPlex-380:~|⇒  docker ps --format "table {{.ID}} \t {{.Names }} \t {{.Status}}"
CONTAINER ID         NAMES               STATUS
7d3a6d518e8e         jenkins             Up 2 minutes
9655eb0e836e         quotes              Up 4 hours

Docker networking

-Mostly used types

Bridge networks
- The default network driver
- Applications run in standalone containers that need to communicate
Overlay networks:
- Connect multiple Docker daemons together
- Enable swarm services to communicate with each other

Docker basic networking

Step 1: create a network and volume

Listing and creating a network:

1 2	docker network ls docker network create net1

Listing and creating a volume:

1 2	docker volume ls docker volume create --driver=local --name=volume1

Step 2: start docker containers

Starting docker containers:

1
2
3

docker run -it --name=container1 -v volume1:/opt --network=net1 busybox sh
docker run -it --name=container2 -v volume1:/opt --network=net1 busybox sh
#-t is used for interactive session and t will give you the terminal

Step 3: test connectivity between these containers

# ping container2
PING container2 (172.19.0.3): 56 data bytes
64 bytes from 172.19.0.3: seq=0 ttl=64 time=0.170 ms
64 bytes from 172.19.0.3: seq=1 ttl=64 time=0.134 ms
64 bytes from 172.19.0.3: seq=2 ttl=64 time=0.133 ms

Overlay networking

What does an overlay network do?

Containers only connect to other containers on the same network
Connecting to containers on other hosts requires that the container publish the needed port
Overlay network allows containers running on different hosts to connect on a private network
Step 1: Initiate docker swarm

Host1: Check networks and initiate docker swarm:
docker network ls
docker swarm init
Host2: Initiate swarm worker and check its networks:
docker swarm join –toaken foobar 192.168.10.91:2377
docker network ls
Step 2: Create overlay network and spin a docker container on host1

Create an overlay network:
docker network create -d overlay –atachable=true swarmnet
- List networks:
  docker network ls
- Spin a Docker container:
  docker run -itd –name container1 –network swarmnet busybox
Step 3: Spin a container on host2 and check its networks

List networks:
docker network ls
- Spin a Docker container:
  docker run -itd –name container2 –network swarmnet busybox
- List networks and see new network:
  docker network ls
Step 4: Login to the containers check its networks and sing each other

Host1: Log in to the container, check IPs, and ping container on Host2:
docker exec -it container1 sh
ifconfig
ping container2
Host2: Log in to the container, check IPs, and ping container on Host1:
docker exec -it container2 sh
ifconfig
ping container1

upload that new image to dockerhub

1 2	docker tag helloworld stanosaka/helloworld docker push stanosaka/helloworld

Cleaning up containers:We can clean up everything by stopping and removing all containers with these two handy one-line commands:

1 2	$ docker stop $(docker ps -a -q) $ docker system prune

Amazon web services and docker development

apt-get update
apt-get install     apt-transport-https     ca-certificates     curl
curl -fsSL https://download.docker.com/linux/ubuntu/gpg | sudo apt-key
add-apt-repository    "deb [arch=amd64] https://download.docker.com/linux/ubuntu \
   $(lsb_release -cs) \
   stable"
apt update
apt-get install docker-ce docker-ce-cli containerd.io
systemctl start docker
systemctl status docker

root@ip-172-31-6-219:~# cat dockerfile
FROM ubuntu:16.04


RUN apt update
RUN apt install -y apache2
RUN echo "Welcome to my web site" > /var/www/html/index.html
CMD /usr/sbin/apache2ctl -D FOREGROUND
EXPOSE 80


docker info
docker run -it --detach=true ubuntu bash
docker ps
docker renmae nervous_visvesvaraya newname
docker network inspect newnet|less
docker network connect newnet newname
docker inspect newname|less
ping 172.18.0.2
ping 172.17.0.2
docker build -t "webserver" .
docker images
docker run -d -p 80:80 webserver /usr/sbin/apache2ctl -D FOREGROUND
docker ps
curl localhost
 aws iam create-service-linked-role --aws-service-name ecs.amazonaws.com
{
    "Role": {
        "AssumeRolePolicyDocument": {
            "Version": "2012-10-17",
            "Statement": [
                {
                    "Action": [
                        "sts:AssumeRole"
                    ],
                    "Effect": "Allow",
                    "Principal": {
                        "Service": [
                            "ecs.amazonaws.com"
                        ]
                    }
                }
            ]
        },
        "RoleId": "AROAYYRUPCKTK7Z2M2PLO",
        "CreateDate": "2019-09-02T02:28:51Z",
        "RoleName": "AWSServiceRoleForECS",
        "Path": "/aws-service-role/ecs.amazonaws.com/",
        "Arn": "arn:aws:iam::602479202982:role/aws-service-role/ecs.amazonaws.com/AWSServiceRoleForECS"
    }
}

Working with ecs throught the aws cli

1
2
3

aws ecs list-clusters
aws ecs describe-clusters
aws ecs create-cluster --cluster-name newcluster

Configure ECS to authenticate with Docker Hub

#1.ssh to ecs server
cd /etc/ecs
[root@ip-172-31-13-18 ecs]# vim ecs.config
ECS_ENGINE_AUTH_TYPE=docker
ECS_ENGINE_AUTH_DATA={"https://index.docker.io/v1/":{"username":"foobar","password":"badpassword","email":"foo.bar@gmail.com"}}
[root@ip-172-31-13-18 ecs]# stop ecs
ecs stop/waiting
[root@ip-172-31-13-18 ecs]# start ecs
ecs start/running, process 4996
[root@ip-172-31-13-18 ecs]# docker inspect ecs-agent|grep ECS_DATADIR
                "ECS_DATADIR=/data",
#2. make repo private in docker hub
#3. configure in aws console about ecs

working with ec2 container registry (ECR)

aws ecr get-login help
docker login -u AWS -p badpassword https://602479202982.dkr.ecr.ap-southeast-2.amazonaws.com
docker images
docker tag c0a44e0f3831 602479202982.dkr.ecr.ap-southeast-2.amazonaws.com/myrepo
docker push 602479202982.dkr.ecr.ap-southeast-2.amazonaws.com/myrepo

Docker registry

docker run -d -p 5000:5000 --restart always --name registry registry:2
docker pull hello-world
docker tag hello-world localhost:5000/hello-world
docker push localhost:5000/hello-world
docker exec -it registry /bin/sh
docker images
docker rmi hello-world
docker rmi localhost:5000/hello-world
docker images
docker pull localhost:5000/hello-world
docker run \
  -d \
  -e ENV_DOCKER_REGISTRY_HOST=registry \
  -e ENV_DOCKER_REGISTRY_PORT=5000 \
  -p 8080:80 \
  --link registry:registry \
  konradkleine/docker-registry-frontend:v2

Error

 docker login 127.0.0.1:5000
Username: myuser
Password:
** Message: 17:25:24.126: Remote error from secret service: org.freedesktop.Secret.Error.IsLocked: Cannot create an item in a locked collection
Error saving credentials: error storing credentials - err: exit status 1, out: `Cannot create an item in a locked collection`

Fix by:

docker logout
Removing login credentials for https://index.docker.io/v1/
➜  outyet git:(master) mv ~/.docker/config.json ~/.docker/config_old.json
➜  outyet git:(master) docke login
zsh: command not found: docke
➜  outyet git:(master) docker login 127.0.0.1:5000
Username: myuser
Password:
WARNING! Your password will be stored unencrypted in /home/stan/.docker/config.json.
Configure a credential helper to remove this warning. See
https://docs.docker.com/engine/reference/commandline/login/#credentials-store

Login Succeeded

Container Security

Containers inside VMs

To get the benefits of containers and VMs, you can run all your containers inside a VM.
This gives you the hypervisor layers you need to separate mission-critical pieces of your app.
Docker, VMware, and other companies build super-fast hypervisor layers for containers.

Some security best practices:

only run verified images so that so attacker can inject their image into your system
have an image registry for your company which let you build, share, track, and run your image.
Image registries like Docker Hub and Docker Trusted Registry implement a tool called Docker Content Trust which lets you sign your images and define exactly what images are allowed to be run on your system.
In addition, there are security scanning tools like Atomic Scan and Docker Security Scanning which you can run on your images once they’ve been built.

These can be automated and implemented as a part of your continuous integration or testing processes.

When creating your images, it’s important that they follow the principle of least privilege.
The container should have the least amount of privileges it needs to do its job.
For many containers, this means implementing read-only file systems. Even if an attacker gets full access to that container, they won’t be able to write anything to the underlying system.

Limit resources for each containers so they can’t be hijacked for other purposes.

Don’t run your processes in the container as a root user.

Use runtime threat detection systems like Aqua Security and Red Hat OpenShfit.

If it seems unusual activity or traffic, you can set up rules so they alert you, shut the system down, or take other steps.

Other Security Measures

Security scanning happends during the build stage, make sure that containers are secure
- Tools include Atomic Scan and Docker Secuiry Scanning.
Create container images following the rule of least privilege
Runtime threat detection uses machine learning to monitor your app during normal running. then detect “abnormal” traffic or activity.
- Tools include Aqua Security and Red Hat OpenShift.

Container Logging and Monitoring

Monitoring the Health of Your System

Lots of orchestration software has monitoring build in.
Kubernetes Heaspster aggregates all K8s data to one spot.
cAdvisor, InfluxDB, and other tools can be used to display data, build graphs, and even build monitoring dashboards from this data.
Third-party cloud services can also plug in and track data from orchestration using APIs.
Offers same alerting, graphing, and dashboard tools as on-prem solutions.
Datadog and Sysdig

Docker + EC2 Variations

Docker out of the box + Vanilla EC2 instances
- Existing docker tools
  - docker-machine, docker swarm, docker-compose, and docker
- Infrastructure as a service (IaaS)
Amazon Elastic Beanstalk + Docker container format
- Beanstalk application delivery
  - Java with Glashfish, Python with uWSGI
- Platform as a service (PaaS)
Docker Datacenter (DDC) for AWS
- Containers as a service (CaaS)
Docker for AWS
ECS managed policies
AmazonEC2ContainerServiceFullAccess
- Added to the ECS Administrator Role
AmazonEC2ContainerServiceforEC2Role
- Added to the ECS Container Instance Role
AmazonEC2ContainerServiceRole
- Added to the ECS Container Scheduler Role
  - Applied to ELB load balancers
  - Register and deregister container instances with load balancers
AmazonEC2ContainerServiceAutoscaleRole
- Added to ECS Auto Scaling Role
  - Used by Application Auto Scale Service
  - Scale service’s desired count in response to CloudWatch alarms

AmazonEC2ContainerServiceTaskRole

Added to ECS Task Role

Used by AWS APIs

Access AWS Resources

virtual machine and Docker container difference

Virtual Machine	Docker Container
Hardware-level process isolation	OS level process isolation
Each VM has a separate OS	Each container can share OS
Boots in minutes	Boots in seconds
VMs are of few GBs	Containers are lightweight (KBs/MBs)
Ready-made VMs are difficult to find	Pre-built docker containers are easily available
VMs can move to new host easily	Containers are destroyed and re-created rather than moving
Creating VM takes a relatively longer time	Containers can be created in seconds
More resource usage	Less resource usage

pipeline

By Stan Zhou

Published 2019-02-24

Pipeline

Represents a part of:

Software delivery
Quality assurance process

Benefits:

Operation grouping
Visibility
Feedback

Automated deployment pipeline
3 stages:
code change-> Continuous Integration-> Automated acceptance testing-> Configuration management

Continuous Integration
The continuous integration phase provides the first feedback to the developers. It checks out the code from the repository, compiles it, runs unit tests and verifies the code quality. If any step fails the pipeline execution is stopped and the first thing the developer should do is fix the continuous integration build. The continuous integration pipeline is usually the starting point.

first feedback
checks code
starting point
simple to setup

Automated acceptance testing

Suites of tests
A quality gate
Pipeline execution is stopped if test fails
Prevents movement
Lot of confusion

Acceptance testing

Represent functional requirements
Wrtten in the form of stories or examples

Unit Testing

Provide the high-quality software
Minimize the number of bugs

Exploratory Testing

Manual black-box testing
Breaks or improves the system

Non-functional testing

Represent properties:
- Performance
- Scalability
- Security

Configuration management

Replaces the manual operations
Responsible for tracking and controlling changes
Solution to the problems
Enable storing configuration files
Configuration management is a solution to the problems posed by manually deploying and configuring applications on the production. Configuration management tools enables storing configuration files in the version control system and tracking every change that was made on the production servers.

Technical and development prerequisites

Automated build, test, package, and deploy operations
Quick pipeline execution
Quick failure recovery
Zero-downtime deployment
Trunk-based deployment

Building the continuous delivery process
Jenkins

Popular automation server
Helps create automated sequence of scripts
Plugin-oriented

Ansible
Helps with:

Software provisioning
Configuration management
Application deployment

Java

Most popular programming language
Develop with the Spring framework
Gradle-build tool
Jenkins is by far the most popular automation server on the market. It helps to create continuous integration and continuous delivery pipelines and in general any other automated sequence of scripts. Highly plug-in oriented it has a greater community which constantly extends it with new features. Another one is Ansible. Ansible is an automation tool that helps with software provisioning configuration management and application deployment. Next comes Java. Java has been the most popular programming language for years, that is why it is being used for most code examples. Together with Java, most companies develop with the Spring framework so we used it to create a simple web service needed to explain some concepts. Gradle is used as a build tool. It is less popular than Maven however trending much faster.

Pipeline elements

Basic elements
Stage - Logical separation
Setp - Used to visualize process

pipeline {
   //agent any
   agent {
      label 'linux'
   }
   stages {
      stage('First Stage') {
          steps {
              echo 'Step 1. Hello World'
          }
      }
      stage('Second Stage') {
          steps {
              echo 'Step 2. Second time Hello'
              echo 'Step 3. Third time Hello'
          }
      }
   }
   post {
     always {
        echo "Pipeline executed!"
     }
   }
}

Define pipeline structure
A declarative pipeline is always specified inside the pipeline block and contains sections, directives and steps.

A declarative pipeline has a simplified and opinionated syntax on top of the pipeline sub-system.

Sections

Keywords are:
- Agent-defines on which agent the pipeline or the stage will be executed
- Stages-defines the stages of the pipeline
- Step
- Post-defines the post build steps

Sections define the pipeline structure and usually contain one or more directives or steps. They are defined with the key words, stages, step and post.

Stages defines a series of one or more stage directives.

Steps defines a series of one or more step instructions.

Steps are the most fundamental part of the pipeline they define the operations that are executed so they actually tell Jenkins what to do.

Defined using:

sh: executes the shell command
custom
script

Posts defines a series of one or more step instructions that are run at the end of the pipeline build.

Directives

Expresses the configuration of a pipeline or its parts
Defined by:
- Agent: specifies where the execution takes place
- Triggers: define automated ways to trigger the pipeline and can use cron to set the time based scheduling
- Options: specified pipeline specific options
- Environment: deinfes a set of key values used a s environment variables
- Parameters: define a list of user input parameters
- Tools: defines the tools configured on Jenkins
- Stage: allows for logical grouping of steps; contains steps and agent
- When: determines whether the stage should be executed depending on the given condition

pipeline {
   agent any
   tools {
      maven 'M3'
   }
  
   parameters {
      string(name: 'VERSION',
          defaultValue: '1.0.0',
          description: 'What is the version to build?')
   }
  
   stages {
      stage('Build'){
          steps {
             sh "./build.sh ${params.VERSION}"
          }
      }
   }
}

Commit Pipeline

Basic continuous integration process
Results in a report about the build
Runs after each change in the code
Consume a reasonable amount of resources
Starting point

Since it runs after every change in the code, the build should take no more than five minutes and should consume a reasonable amount of resources. The commit phase is always the starting point of the continuous delivery process and it provides the most important feedback cycle in the development process.

In the commit phase a developer checks in the code to the repository.
The continuous integration server detects the change and the build starts.
The most fundamental commit pipeline contains three stages

checkout
compile
unit test

Code Quality Stages

Extending continuous integration

The most widely used are: Code coverage and Static analysis

Code coverage

Scenario:
- Nobody writes unit tests
- Passes all the builds
Solution:
- Add coverage code tools
Creates report
Make build fail

Available tools

JaCoCo
Clover
Cobertura

Static Code Analysis

Checks without execution
Checks number of rules
Rules apply to wide range of aspects
Popular tools are Checkstyle, FindBugs, and PMD

SonarQube

Quality management tool
An alternative
Aggregates different code analysis frameworks
Has own dashboards
User friendly web interface

Triggers and notifications

Triggers

Automatic action to start the build
Manay options to choose from
Three types:
- External
- Polling SCM
- Scheduled build

External Trigger

Natural to understand
Starts the build after it’s called by the notifier

GitHub -> trigger-> Jenkins

Polling SCM

Periodically calls Github
Sound counter-intuitive
Polling SCM trigger is less intuitive.

Jenkins periodically calls GitHub and checks if there were any push to the repository. Then it starts the build.

Scheduled build

Jenkins runs the build periodically
No communication with any system
Implementation of scheduled build is same as polling SCM
Used for the commit pipeline

Notifications

Team development strategies

Development workflows

Put the code into the repostitory
Depends on many factors
Classify into three types:

Feature toggle
Feature toggle is a technique that is an alternative to maintaining multiple source code branches such that the feature can be tested before it is completed and ready for use. It is used to disable the feature for users but enables it for developers while testing. Feature toggles are essentially variables used in conditional statements the simplest implementation of feature toggles are flags and the if statements.

acceptance testing
Acceptance testing is a test performed to determine if business requirements or contracts are met.

Invovles black-box testing
Imply the acceptance of the software delivery
Also called UAT
Rely on manual steps
Reaonable to run them as programmed repeatable operations

Artifact repository
While the source control management stores the source code the artifact repository is dedicated for storing software binary artifacts, for example, compiled libraries or components later used to build a complete application.

Store binaries on a separate server due to:
File size
Versions
Revision mapping
Packages
Access Control
Clients
Use cases

private docker registry

mkdir -p certs

openssl req \\n  -newkey rsa:4096 -nodes -sha256 -keyout certs/domain.key \\n  -x509 -days 365 -out certs/domain.crt

docker run -d -p 5000:5000 --restart=always --name registry -v `pwd`/auth:/auth -e "REGISTRY_AUTH=htpasswd" -e "REGISTRY_AUTH_HTPASSWD_REALM=Registry Realm" -e REGISTRY_AUTH_HTPASSWD_PATH=/auth/passwords -v `pwd`/certs:/certs -e REGISTRY_HTTP_TLS_CERTIFICATE=/certs/domain.crt -e REGISTRY_HTTP_TLS_KEY=/certs/domain.key registry:2

docker tag ubuntu_with_python stanosaka/ubuntu_with_python:1
docker push stanosaka/ubuntu_with_python:1

Acceptance test in pipeline

The developer pushes a code change to Github.
Jenkins detects the change, triggers the build and checks out the current code.
Jenkins executes the commit phase and builds the Docker image
Jenkins pushes the image to Docker Registry
Jenkins runs the Docker container in the staging environment

Configuration management

Process of controlling configuration changes
Used to refer to the software and the hardware

Application Configuration| Infrastructre Configuration
Decides how the system works| Server infrastructure and environment configuration
Expressed in the form of flags| Takes care of the deployment process

Traits

Automation
Version Control
Incremental changes
Server provisioning
Security
Simplicity

kubernetes on the cloud

By Stan Zhou

Published 2019-02-01

Kubernetes on the GKE

use cloud shell

#setup zone
gcloud config set compute/zone australia-southeast1-a
#setup region
gcloud config set compute/region australia-southeast1
# create a single node cluster
gcloud container clusters create my-first-cluster --num-nodes 1
# see the list of clusters
gcloud compute instances list
# deploy a wordpress Docker container to our single node cluster
# --image=tutum/wordpress: An out-fo-the-box image which includes everything you need
# to run this site-including a MySQL database
kubectl run wordpress --image=tutum/wordpress --port=80
# see the status of pods running
stancloud9@cloudshell:~ (scenic-torch-250909)$ kubectl get pods
NAME                         READY   STATUS    RESTARTS   AGE
wordpress-7fff795d48-kzbfr   1/1     Running   0          5m36s

By default a pod is accessible to only other internal machines in the cluster

#kubectl expose pod: Exposed the pod a a service so it can be accessed externally
#--type=LoadBalancer: Creates an external IP that this pod can use to accept traffic
kubectl expose pod wordpress-7fff795d48-kzbfr --name=wordpress --type=LoadBalancer
# check 
stancloud9@cloudshell:~ (scenic-torch-250909)$ kubectl describe svc wordpress
Name:                     wordpress
Namespace:                default
Labels:                   pod-template-hash=7fff795d48
                          run=wordpress
Annotations:              <none>
Selector:                 pod-template-hash=7fff795d48,run=wordpress
Type:                     LoadBalancer
IP:                       10.27.245.246
LoadBalancer Ingress:     35.189.22.57
Port:                     <unset>  80/TCP
TargetPort:               80/TCP
NodePort:                 <unset>  31344/TCP
Endpoints:                10.24.0.11:80
Session Affinity:         None
External Traffic Policy:  Cluster
Events:
  Type    Reason                Age    From                Message
  ----    ------                ----   ----                -------
  Normal  EnsuringLoadBalancer  2m17s  service-controller  Ensuring load balancer
  Normal  EnsuredLoadBalancer   81s    service-controller  Ensured load balancer

How Kubernetes Works
Kubernetes: Orchestration technology- convert isolated containers running on different hardware into a cluster

Pod: Atomic unit of deployment in Kubernetes.

Consists of 1 or more tightly coupled containers.
Pod runs on node, which is controlled by master.
Kubernetes only knows about pods
Cannot start container without a pod
Pod => Sandbox for 1 or more cntainers

Kubernetes::Hadoop

Docker Container Engine -> Java Runtime
Docker containers -> Jars
Kubernetes -> Hadoop

Kubernetes for Orchestration

Fault-tolerance: Pod/Nod failures
Auto-scaling: More clients? More demand
Rollback: Advanced deployment options
Auto-healing: Crashed containers restart
Load-balancing: Distribute client requests
Isolation: Sandboxes so that containers don’t interfere

kubectl

Telling k8s what the desire state is.

What does the k8s master do?
Kubernetes Master

One or more nodes designated as master
Several k8s processes run on master
Multi-master for high-availability

kube-apiserver

Communicates with user
RESTful API end-points
Manifest yaml files are accepted by apiserver

etcd
Cluster Store for Metadata

Metadata about spec and status of cluster
etcd is consistent and highly available key-value store
source-of-truth for cluster state

kube-scheduler

Handle pod creation and management
Kube-scheduler match/assign nodes to pods
Complex-affinities, taints, tolerations,…

controller-manager

Different master processes
Actual state <-> Desired State
cloud-controller-manager
kube-controller-manager
Node controller
Replication controller
Route controller
Volume controller

Control Plane

Apiserver
etc
scheduler
controller-manager
- cloud-controller-manager
- kube-controller-manager

What runs on each node of a cluster?
Kubernetes Node (Minion)

Kubelet
- Agent running on this node
- Listen to k8s master
- Port 10255
Kube-proxy
- Needed because pod IP addresses are ephemeral
- Networking- will make sense when we discuss Service objects
Container engine
- Works with kubelet
- Pulling images
- Start/stop
- Could be Docker or rkt

What are Pods?

Multi-Container Pods

Share access to memory space
Connect to each other using localhost
Share access to the same volumes (storage abstraction)
Same parameters such as configMaps
Tight coupling is dangerous
Once crashes, all crash

Use cases for multi-container Pod

Main container, “sidecar” supporting containers
Proxies, Bridges, Adapters

Anti-Patterns for multi-container pods

Avoid packing three-tier web app into 1 pod
Do not pack multiple similar containers in the same pod for scaling
Use Deployments or ReplicaSet instead
Micro0services: Simple, independent components

Pods limitations

No auto-healing or scaling
Pod crashes? Must be handled at higher level
- ReplicaSet,Deployment,Service
Ephemeral: Ip address are ephemeral

Higer level k8s objects

ReplicaSet, ReplicationController: Scaling and healing
Deployment: Versioning and rollback
Service: Static (non-ephemeral) IP and networking
Volume: Non-ephemeral storage
How do master nodes communicate?

Cluster to master

All cluster -> master communication only with apiserver
HTTPS (443)
Relatively secure

Master to cluster

apiserver->kubelet
- Certicate not verified by default
- Vulnerable to man-in-the-middle attacks
- Don’t run on public network
- To harden
  - set -kubelet-certificate-authority
  - use SSH tunelling
apiserver->nodes/pods/services
- Not safe
- Plain HTTP
- Neither authenticated for encrypted
- On public clouds, SSH tunnelling provided by cloud provider e.g. GCP
  Where can we run Kubernetes?
  Public Cloud:
aws(kops)
azure
gcp(kubectl)
- GKE:Google Kubernetes Engine
- Infra on GCE Virtual machines
- GCE: Google Compute Engine (IaaS)
  Bootstrap(kubeadm): On-prem, private cloud
  Playgrounds:PWK(Browser-based,time-limited sessions), Minikube(Windows or Mac, Sets up VM on your machine)

Can K8s be used in a hybird, multi-cloud world?

Hybrid=On-prem+Public Cloud

On-premise: vare metal or VMs
Legacy infra, large on-prem datacenters
Medium-term importance

Multi-Cloud

More than 1 public cloud provider
Strategic reasons, vendor lock-in (Amazon buying Whole Foods)

Interacting with K8s

How do we work with k8s

kubectl
- Most common command line utility
- Make POST requests to apiserver of control plane
kubeadm
- Bootstrap cluster when not on cloud Kubernetes service
- To create cluster out of individual infra nodes
kubefed
- Administer federated clusters
- Federated cluster-> group of multiple clusters (multi-cloud,hybrid)
kubelet
kube-proxy

Objects

K8s objects are persistent entities
Everything is an object…
Pod,ReplicaSet, Deployment, Node … all are objects
Send object specification (usually in .yaml or .json)

Three object management methods

Imperative commands
- No .yaml or config files
- kubectl run …
- kubectl expose …
- kubectl autosacle …
  1
  2
  kubectl run ningx --image nginx
  kubectl create deployment nginx --image nginx
  No config file

Imperative: intent is in command

Pro:

Simple
Cons:
No audit trail or review mechanism
Can’t reuse or use in template

Imperative object configuration
- kubectl + yaml or config files used
- kubectl create -f config.yaml
- kubectl replace -f config.yaml
- kubectl delete -f config.yaml

Config file required
Still Imperative: intent is in command

Pros:

Still simple
Robust - files checked into repo
One file for multiple operations

Declarative object configuration
- Only .yaml or config files used
- kubectl apply -f config.yaml
  1
  kubectl apply -f configs/
  Config files(s) are all that is required
  Declarative not imperative

Pros:

Most robust-review, repos, audit trails
K8S will automatically figure out intents
Can specify multiple files/directories recursively
Live object configuration: The live configuration values of an object, as observed by the Kubernetes cluster
Current object configuration files: The config file we are applying in the current command
List-applied object configuration file: The last config file what was applied to the object

Don’t mix and match!
Declarative is preferred

Merging Changes

Primitive fields
- String, init, boolean, images or replicas
- Replace old state with Current object configuration file
Map fields
- Merge old state with Current object configuration file
List field
- Complex-varies by field

The Pros and Cons of Declarative and Imperative object management
Declarative

kubectl apply -f config.yaml

Robust
Track in repos, review
Recursively apply to directories

Imperative
kubectl run …
kubectl expose…
kubectl autosacle…

Simple, intention is very clear
Preferred for deletion

Names and UIDs

How are objects named?

Objects-persistent entities
- pods,replicasets,services,volumes,nodes,…
- information maintained in etcd
Identified using
- names: client-given
- UIDs: system-generated
  Namespaces
Divide physical cluster into multiple virtual clusters
Three pre-defined namespaces:
- default: if non specified
- kube-system: for internal k8s objects
- kub-public: auto-readable by all users
Name scopes: names need to be unique only inside a namespace
Future version: namespace-> common access control
Don’t use namespaces for versioning
- Just use labels instead

Objects without namespaces

Nodes
PersistentVolumes
Namespace themselves

Labels

key/value pairs attached to objects
metadata
need not be unique (same label to mutiple objects)
No sematics for k8s (meaningful only to humans)
Loose coupling via selectors
Can be added
- at creation time
- after creation
Multiple objects can have same label
Same label can’t repeat key though

Volumes

Permanence: Storage that lasts longer than lifetime of a pod
- a container inside pod (true for all volumes)
- a pod (only true for persistent volumes)
Shared State: multiple containers in a pod need to share state/files
Volumes: address both these needs
Volumes (in general): lifetime of abstraction=lifetime of pod
- Note that this is longer than lifetime of any container inside pod
Persistent Volumes: lifetime of abstraction independent of pod lifetime

Types of Volumes

awsElasticBlockStore
azureDisk
azureFile
gcePersistentDisk
Many non-cloud-specific volume types as well

Important type of volumes

configMap
emptyDir
gitRepo
secret

hostPath

Lab: Volumes and the exmptydir volume

stancloud9@cloudshell:~ (scenic-torch-250909)$ cat pod-redis.yaml
apiVersion: v1
kind: Pod
metadata:
   name: redis
spec:
   containers:
   - name: redis
     image: redis
     volumeMounts:
     - name: redis-storage
       mountPath: /data/redis
   volumes:
   - name: redis-storage
     emptyDir: {}
stancloud9@cloudshell:~ (scenic-torch-250909)$ k get pod redis --watch        NAME    READY   STATUS    RESTARTS   AGE
redis   0/1     Pending   0          7m36s
redis   0/1   Pending   0     7m36s
redis   0/1   ContainerCreating   0     7m36s
redis   1/1   Running   0     7m40s
^Cstancloud9@cloudshell:~ (scenic-torch-250909)$ k exec -it redis -- /bin/bash
root@redis:/data# cd /data/redis
root@redis:/data/redis# echo Hello > test-file
root@redis:/data/redis# kill 1
root@redis:/data/redis# command terminated with exit code 137
stancloud9@cloudshell:~ (scenic-torch-250909)$ k get pod redis --watch
NAME    READY   STATUS    RESTARTS   AGE
redis   1/1     Running   1          18m
^Cstancloud9@cloudshell:~ (scenic-torch-250909)$ k exec -it redis -- /bin/bash
root@redis:/data# ls /data/redis
test-file

Persistent Volumes

Low-level objects, like nodes
Two types of provisioning:
Static: administrator pre-creates the volumes

Dynamic: containers need to file a PersistentVolumeClaim

gcloud compute disks create my-disk-1 --zone australia-southeast1-a
gcloud compute disks list
stancloud9@cloudshell:~ (scenic-torch-250909)$ cat volum-sample.yaml
apiVersion: v1
kind: Pod
metadata:
  name: test-pd
spec:
  containers:
  - image: k8s.gcr.io/test-webserver
    name: test-container
    volumeMounts:
    - mountPath: /test-pd
      name: test-volume
  volumes:
  - name: test-volume
    gcePersistentDisk:
      pdName: my-disk-1
      fsType: ext4
k create -f volum-sample.yaml --validate=false
pod/test-pd created
stancloud9@cloudshell:~ (scenic-torch-250909)$ k get pod test-pd --watch
NAME      READY   STATUS              RESTARTS   AGE
test-pd   0/1     ContainerCreating   0          23s
test-pd   1/1   Running   0     29s
^Cstancloud9@cloudshell:~ (scenic-torch-250909)$ k describe pod test-pd
Name:               test-pd
Namespace:          default
Priority:           0
PriorityClassName:  <none>
Node:               gke-my-first-cluster-default-pool-8ca613e9-w13m/10.152.0.2
Start Time:         Sat, 31 Aug 2019 11:39:22 +1000
Labels:             <none>
Annotations:        kubernetes.io/limit-ranger: LimitRanger plugin set: cpu request for container test-container
Status:             Running
IP:                 10.24.0.22
Containers:
  test-container:
    Container ID:   docker://8b08b4cd144d256a078fe3c7b4031844e2c3355d854a711ec454611c0edea30b
    Image:          k8s.gcr.io/test-webserver
    Image ID:       docker-pullable://k8s.gcr.io/test-webserver@sha256:f63e365c13646f231ec4a16791c6133ddd7b80fcd1947f41ab193968e02b0745
    Port:           <none>
    Host Port:      <none>
    State:          Running
      Started:      Sat, 31 Aug 2019 11:39:50 +1000
    Ready:          True
    Restart Count:  0
    Requests:
      cpu:        100m
    Environment:  <none>
    Mounts:
      /test-pd from test-volume (rw)
      /var/run/secrets/kubernetes.io/serviceaccount from default-token-gw57h (ro)
Conditions:
  Type              Status
  Initialized       True
  Ready             True
  ContainersReady   True
  PodScheduled      True
Volumes:
  test-volume:
    Type:       GCEPersistentDisk (a Persistent Disk resource in Google Compute Engine)
    PDName:     my-disk-1
    FSType:     ext4
    Partition:  0
    ReadOnly:   false
  default-token-gw57h:
    Type:        Secret (a volume populated by a Secret)
    SecretName:  default-token-gw57h
    Optional:    false
QoS Class:       Burstable
Node-Selectors:  <none>
Tolerations:     node.kubernetes.io/not-ready:NoExecute for 300s
                 node.kubernetes.io/unreachable:NoExecute for 300s
Events:
  Type    Reason                  Age    From                                                      Message
  ----    ------                  ----   ----                                                      -------
  Normal  Scheduled               2m15s  default-scheduler                                         Successfully assigned default/test-pd to gke-my-first-cluster-default-pool-8ca613e9-w13m
  Normal  SuccessfulAttachVolume  2m9s   attachdetach-controller                                   AttachVolume.Attach succeeded for volume "test-volume"
  Normal  Pulling                 109s   kubelet, gke-my-first-cluster-default-pool-8ca613e9-w13m  pulling image "k8s.gcr.io/test-webserver"
  Normal  Pulled                  107s   kubelet, gke-my-first-cluster-default-pool-8ca613e9-w13m  Successfully pulled image "k8s.gcr.io/test-webserver"
  Normal  Created                 107s   kubelet, gke-my-first-cluster-default-pool-8ca613e9-w13m  Created container
  Normal  Started                 107s   kubelet, gke-my-first-cluster-default-pool-8ca613e9-w13m  Started container

What are some important types of volumes?

emptyDir
- Not persistent
- Created when pod is created on node
- Initially empty
- Share space/state across containers in same pod
- Containers acan mount at different times
- When pod removed/crashes, data lost
- When container carshes data remains
- Usecases: Scratch space, checkpointing…
hostPath
- Mount file/directory from node filesystem into pod
- Uncommon- pods should be independent of nodes
- Makes pod-node coupling tight
- Usecases: Access docker internals, running cAdvisor
- Block devices or sockets on host
gitRepo
configMap
- configMap volume mounts data from ConfigMap object
- configMap objects define key-value pairs
- configMap objects inject paramters into pods
- Two main usecases:
  - Providing config information for apps running inside pods
  - Specifying config information for control plane (controllers)

secret

Pass ensitive inforamtion to pods
First create secret so it is stored in control plane
- kubectl create secret
Mount that secret as a volume so that it is available inside pod

Secret is stored in RAM storage (not written to persistent disk)

Lab: use of secrets pass information to pods

stancloud9@cloudshell:~ (scenic-torch-250909)$ cat secrets.yaml
apiVersion: v1
kind: Secret
metadata:
  name: test-secret
data:
  username: bXktYMxw
  password: Mzk1MjqkdmRnN0pi
stancloud9@cloudshell:~ (scenic-torch-250909)$ cat secrets-pod.yaml
apiVersion: v1
kind: Pod
metadata:
  name: secret-test-pod
spec:
  containers:
    - name: test-container
      image: nginx
      volumeMounts:
         # name must match the volume name below
         - name: secret-volume
           mountPath: /etc/secret-volume
  # The secret data is exposed to Containers in the Pod through a Volume.
  volumes:
    - name: secret-volume
      secret:
        secretName: test-secret
stancloud9@cloudshell:~ (scenic-torch-250909)$ k get pod secret-test-pod
NAME              READY   STATUS    RESTARTS   AGE
secret-test-pod   1/1     Running   0          19m
stancloud9@cloudshell:~ (scenic-torch-250909)$ k exec -it secret-test-pod -- /bin/bash
root@secret-test-pod:/# cd /etc/secret-volume/
root@secret-test-pod:/etc/secret-volume# ls
password  username

Debugging

Insufficient cpu

stancloud9@cloudshell:~ (scenic-torch-250909)$ k get pod secret-test-pod
NAME              READY   STATUS    RESTARTS   AGE
secret-test-pod   0/1     Pending   0          51s
stancloud9@cloudshell:~ (scenic-torch-250909)$ k describe pod secret-test-pod
Name:               secret-test-pod
Namespace:          default
Priority:           0
PriorityClassName:  <none>
Node:               <none>
Labels:             <none>
Annotations:        kubernetes.io/limit-ranger: LimitRanger plugin set: cpu request for container test-container
Status:             Pending
IP:
Containers:
  test-container:
    Image:      nginx
    Port:       <none>
    Host Port:  <none>
    Requests:
      cpu:        100m
    Environment:  <none>
    Mounts:
      /etc/secret-volume from secret-volume (rw)
      /var/run/secrets/kubernetes.io/serviceaccount from default-token-gw57h (ro)
Conditions:
  Type           Status
  PodScheduled   False
Volumes:
  secret-volume:
    Type:        Secret (a volume populated by a Secret)
    SecretName:  test-secret
    Optional:    false
  default-token-gw57h:
    Type:        Secret (a volume populated by a Secret)
    SecretName:  default-token-gw57h
    Optional:    false
QoS Class:       Burstable
Node-Selectors:  <none>
Tolerations:     node.kubernetes.io/not-ready:NoExecute for 300s
                 node.kubernetes.io/unreachable:NoExecute for 300s
Events:
  Type     Reason            Age                  From               Message
  ----     ------            ----                 ----               -------
  Warning  FailedScheduling  11s (x8 over 4m25s)  default-scheduler  0/1 nodes are available: 1 Insufficient cpu.

stancloud9@cloudshell:~ (scenic-torch-250909)$ gcloud container clusters resize my-first-cluster --num-nodes=3 --zone australia-southeast1-a
Pool [default-pool] for [my-first-cluster] will be resized to 3.

Do you want to continue (Y/n)?  Y

Resizing my-first-cluster...done.
Updated [https://container.googleapis.com/v1/projects/scenic-torch-250909/zones/australia-southeast1-a/clusters/my-first-cluster].
stancloud9@cloudshell:~ (scenic-torch-250909)$ k get pod secret-test-pod
NAME              READY   STATUS    RESTARTS   AGE
secret-test-pod   1/1     Running   0          19m

redhat

By Stan Zhou

Published 2017-09-06

Issue 1.

[root@localhost stan]# yum install -y ntp
Loaded plugins: langpacks, product-id, search-disabled-repos, subscription-manager
This system is not registered to Red Hat Subscription Management. You can use subscription-manager to register.
There are no enabled repos.
 Run "yum repolist all" to see the repos you have.
 You can enable repos with yum-config-manager --enable <repo>

Solve:

[root@localhost yum.repos.d]# mkdir /cdrom
[root@localhost yum.repos.d]# mount /dev/cdrom /cdrom
[root@localhost yum.repos.d]# pwd
/etc/yum.repos.d
[root@localhost yum.repos.d]# vim dvd.repo
[dvd-source]
name=RHEL 7.2 dvd repo
baseurl=file:///cdrom
enabled=1
gpgcheck=0
[root@localhost yum.repos.d]# yum repolist
Loaded plugins: langpacks, product-id, search-disabled-repos, subscription-manager
This system is not registered to Red Hat Subscription Management. You can use subscription-manager to register.
dvd-source                                                                                     | 4.1 kB  00:00:00
(1/2): dvd-source/group_gz                                                                     | 136 kB  00:00:00
(2/2): dvd-source/primary_db                                                                   | 3.6 MB  00:00:00
repo id                                             repo name                                                   status
dvd-source                                          RHEL 7.2 dvd repo                                           4,620
repolist: 4,620
[root@localhost yum.repos.d]# yum install rpm-build
Loaded plugins: langpacks, product-id, search-disabled-repos, subscription-manager
This system is not registered to Red Hat Subscription Management. You can use subscription-manager to register.
Resolving Dependencies
--> Running transaction check
---> Package rpm-build.x86_64 0:4.11.3-17.el7 will be installed
--> Processing Dependency: perl(Thread::Queue) for package: rpm-build-4.11.3-17.el7.x86_64
--> Processing Dependency: system-rpm-config for package: rpm-build-4.11.3-17.el7.x86_64
--> Running transaction check
---> Package perl-Thread-Queue.noarch 0:3.02-2.el7 will be installed
---> Package redhat-rpm-config.noarch 0:9.1.0-68.el7 will be installed
--> Processing Dependency: dwz >= 0.4 for package: redhat-rpm-config-9.1.0-68.el7.noarch
--> Processing Dependency: perl-srpm-macros for package: redhat-rpm-config-9.1.0-68.el7.noarch
--> Running transaction check
---> Package dwz.x86_64 0:0.11-3.el7 will be installed
---> Package perl-srpm-macros.noarch 0:1-8.el7 will be installed
--> Finished Dependency Resolution

Dependencies Resolved

======================================================================================================================
 Package                          Arch                  Version                       Repository                 Size
======================================================================================================================
Installing:
 rpm-build                        x86_64                4.11.3-17.el7                 dvd-source                144 k
Installing for dependencies:
 dwz                              x86_64                0.11-3.el7                    dvd-source                 99 k
 perl-Thread-Queue                noarch                3.02-2.el7                    dvd-source                 17 k
 perl-srpm-macros                 noarch                1-8.el7                       dvd-source                4.7 k
 redhat-rpm-config                noarch                9.1.0-68.el7                  dvd-source                 77 k

Transaction Summary
======================================================================================================================
Install  1 Package (+4 Dependent packages)

Total download size: 341 k
Installed size: 734 k
Is this ok [y/d/N]: y
Downloading packages:
----------------------------------------------------------------------------------------------------------------------
Total                                                                                 1.9 MB/s | 341 kB  00:00:00
Running transaction check
Running transaction test
Transaction test succeeded
Running transaction
  Installing : perl-Thread-Queue-3.02-2.el7.noarch                                                                1/5
  Installing : dwz-0.11-3.el7.x86_64                                                                              2/5
  Installing : perl-srpm-macros-1-8.el7.noarch                                                                    3/5
  Installing : redhat-rpm-config-9.1.0-68.el7.noarch                                                              4/5
  Installing : rpm-build-4.11.3-17.el7.x86_64                                                                     5/5
dvd-source/productid                                                                           | 1.6 kB  00:00:00
  Verifying  : redhat-rpm-config-9.1.0-68.el7.noarch                                                              1/5
  Verifying  : rpm-build-4.11.3-17.el7.x86_64                                                                     2/5
  Verifying  : perl-srpm-macros-1-8.el7.noarch                                                                    3/5
  Verifying  : dwz-0.11-3.el7.x86_64                                                                              4/5
  Verifying  : perl-Thread-Queue-3.02-2.el7.noarch                                                                5/5

Installed:
  rpm-build.x86_64 0:4.11.3-17.el7

Dependency Installed:
  dwz.x86_64 0:0.11-3.el7                  perl-Thread-Queue.noarch 0:3.02-2.el7  perl-srpm-macros.noarch 0:1-8.el7
  redhat-rpm-config.noarch 0:9.1.0-68.el7

omplete!

$(!!) to recompute the output of the last command

Nginx Deploy

By Stan Zhou

Published 2016-10-27

Install Nginx

Install Nginx Package

[root@huoq bin]# yum install pcre pcre-devel -y
check by:
[root@huoq bin]# rpm -qa pcre pcre-devel
pcre-devel-8.32-15.el7_2.1.x86_64
pcre-8.32-15.el7_2.1.x86_64
[root@huoq tools]# yum install -y openssl-devel openssl

Install Nginx

wget http://nginx.org/download/nginx-1.6.3.tar.gz
useradd nginx -s /sbin/nologin -M
tar xf nginx-1.6.3.tar.gz
cd nginx-1.6.3/
./configure --user=nginx --group=nginx --prefix=/application/nginx-1.6.3 --with-http_stub_status_module --with-http_ssl_module
make
make install
ln -s /application/nginx-1.6.3 /application/nginx
vim /application/nginx/conf/nginx.conf
worker_processes  4;

events {
    worker_connections  1024;
}

http {
    include       mime.types;
    default_type  application/octet-stream;
    log_format  main  '$remote_addr - $remote_user [$time_local] "$request" '
                      '$status $body_bytes_sent "$http_referer" '
                      '"$http_user_agent" "$http_x_forwarded_for"';

    access_log  logs/access.log  main;

    sendfile        on;
   
    keepalive_timeout  65;

    upstream web_pools {
        server 127.0.0.1:8081;
        server 127.0.0.1:8082;
        }

    server {
        listen       80;
        server_name  localhost;

        location / {
            root   html;
            index  index.html index.htm;
            proxy_pass http://web_pools;
        }
        location /nginx_status {
                stub_status on;
                access_log off;
                allow 127.0.0.1;
                deny all;
        }
       
        # redirect server error pages to the static page /50x.html
        #
        error_page   500 502 503 504  /50x.html;
        location = /50x.html {
            root   html;
        }
    }
}
/application/nginx/sbin/nginx -t
vim /etc/init.d/nginx
#!/bin/bash
#
#chkconfig: - 85 15
#description: this script use to manage nginx process.
#

#set -x
. /etc/rc.d/init.d/functions

procnum=`ps -ef |grep "/application/nginx/sbin/nginx"|grep -v "grep"|wc -l`

start () {
  if [ "$procnum" -eq 1 -a -f /application/nginx/logs/nginx.pid ]; then
  echo -n "Starting nginx:"
  success
  echo
  else
  /application/nginx/sbin/nginx
  if [ "$?" -eq 0 ]; then
  echo -n "Starting nginx:"
  success
  echo
  else
  echo -n "Starting nginx:"
  failure
  echo
  exit 4
  fi
  fi
}

stop () {
  if [ "$procnum" -eq 1 -a -f /application/nginx/logs/nginx.pid ]; then
  /application/nginx/sbin/nginx -s stop
  if [ "$?" -eq 0 ]; then
  echo -n "Stopping nginx:"
  success
  echo
  else
  echo -n "Stopping nginx:"
  failure
  echo
  exit 3
  fi
  else
  echo -n "Stopping nginx:"
  success
  echo
  fi
}

case $1 in

  start)
  start
  ;;

  stop)
  stop
  ;;

  restart)
  stop
  sleep 1
  start
  ;;

  reload)
  if [ "$procnum" -eq 1 -a -f /application/nginx/logs/nginx.pid ]; then
  /application/nginx/sbin/nginx -s reload
  else
  echo "nginx is not running!please start nginx first..."
  exit 2
  fi
  ;;

  status)
  if [ "$procnum" -eq 1 -a -f /application/nginx/logs/nginx.pid ]; then
  echo "nginx is running..."
  else
  echo "nginx is not running..."
  fi
  ;;

  *)
  echo "Usage : nginx [ start|stop|reload|restart|status ]"
  exit 1
  ;;
esac
chmod +x /etc/init.d/nginx
chkconfig nginx on
/etc/init.d/nginx start
Starting nginx:                                      [  OK  ]
/etc/init.d/nginx status
nginx is running...

Written with StackEdit.

tomcat deploy

By Stan Zhou

Published 2016-10-27

tomcat deploy on centos

软件准备

JDK下载：

1	wget --no-check-certificate --no-cookies --header "Cookie: oraclelicense=accept-securebackup-cookie" http://download.oracle.com/otn-pub/java/jdk/7u79-b15/jdk-7u79-linux-x64.tar.gz

Tomcat下载:

1	wget ftp://toad.cwzhou.win/pub/apache-tomcat-7.0.70.tar.gz

部署java环境

tar xf jdk-8u91-linux-x64.tar.gz -C /application/

ln -s /application/jdk1.8.0/ /application/jdk
cp /etc/profile /etc/profile.2016.6.29
# sed -i.ori '$a export JAVA_HOME=/application/jdk\nexport PATH=$JAVA_HOME/bin:$JAVA_HOME/jre/bin:$PATH\nexport CLASSPATH=.$CLASSPATH:$JAVA_HOME/lib:$JAVA_HOME/jre/lib:$JAVA_HOME/lib/tools.jar' /etc/profile

vimdiff /etc/profile /etc/profile.2016.6.29
source /etc/profile
java -version

Install tomcat

tar xf apache-tomcat-8.0.27.tar.gz -C /application/
ln -s /application/apache-tomcat-8.0.27 /application/tomcat
echo  'export TOMCAT_HOME=/application/tomcat'>>/etc/profile

source /etc/profile
chown -R root.root /application/jdk/ /application/tomcat/
[root@tomcat ~]# tail -4 /etc/profile
export JAVA_HOME=/application/jdk
export PATH=$JAVA_HOME/bin:$JAVA_HOME/jre/bin:$PATH
export CLASSPATH=.$CLASSPATH:$JAVA_HOME/lib:$JAVA_HOME/jre/lib:$JAVA_HOME/lib/tools.jar
export TOMCAT_HOME=/application/tomcat

Tomcat目录介绍

[root@tomcat ~]# cd /application/tomcat/
[root@tomcat tomcat]# tree -L 1
.
├── bin #→用以启动、关闭Tomcat或者其它功能的脚本（.bat文件和.sh文件）
├── conf #→用以配置Tomcat的XML及DTD文件
├── lib #→存放web应用能访问的JAR包
├── LICENSE
├── logs #→Catalina和其它Web应用程序的日志文件
├── NOTICE
├── RELEASE-NOTES
├── RUNNING.txt
├── temp #→临时文件
├── webapps #→Web应用程序根目录
└── work #→用以产生有JSP编译出的Servlet的.java和.class文件
7 directories, 4 files
[root@tomcat tomcat]# cd webapps/
[root@tomcat webapps]# ll
total 20
drwxr-xr-x 14 root root 4096 Oct 5 12:09 docs #→tomcat帮助文档
drwxr-xr-x 6 root root 4096 Oct 5 12:09 examples #→web应用实例
drwxr-xr-x 5 root root 4096 Oct 5 12:09 host-manager #→管理
drwxr-xr-x 5 root root 4096 Oct 5 12:09 manager #→管理
drwxr-xr-x 3 root root 4096 Oct 5 12:09 ROOT #→默认网站根目录

Tomcat多实例及集群架构

Tomcat多实例

复制Tomcat目录

1
2
3

[root@tomcat ~]# cd /application/
 [root@yoshi application]# cp -a apache-tomcat-7.0.70 tomcat7_1
 [root@yoshi application]# cp -a apache-tomcat-7.0.70 tomcat7_2

修改配置文件

[root@tomcat application]# mkdir -p /data/www/www/ROOT
[root@tomcat application]# cp /application/tomcat/webapps/memtest/meminfo.jsp /data/www/www/ROOT/
sed -i '22s#8005#8011#;71s#8080#8081#;125s#appBase=".*"# appBase="/data/www/www"#' /application/tomcat7_1/conf/server.xml

sed -i '22s#8005#8012#;71s#8080#8082#;125s#appBase=".*"# appBase="/data/www/www"#' /application/tomcat7_2/conf/server.xml

jvm调优

1 2	# vim catalina.sh insert JAVA_OPTS="-Djava.awt.headless=true -Dfile.encoding=UTF-8 -server -Xms1024m -Xmx1024m -XX:NewSize=512m -XX:MaxNewSize=512m -XX:PermSize=512m -XX:MaxPermSize=512m"

service startup script

cd /etc/init.d

vim tomcat
#!/bin/bash
# des: Tomcat Start Stop Restart
# processname: tomcat
# chkconfig: 234 20 80
JAVA_HOME=/application/jdk
export JAVA_HOME
PATH=$JAVA_HOME/bin:$PATH
export PATH
CATALINA_HOME=/application/tomcat

case $1 in
start)
sh /application/tomcat7_1/bin/startup.sh
sh /application/tomcat7_2/bin/startup.sh
;;
stop)
sh /application/tomcat7_1/bin/shutdown.sh
sh /application/tomcat7_2/bin/shutdown.sh
;;
restart)
sh /application/tomcat7_1/bin/shutdown.sh
sh /application/tomcat7_1/bin/startup.sh
sh /application/tomcat7_2/bin/shutdown.sh
sh /application/tomcat7_2/bin/startup.sh
;;
esac
exit 0

chkconfig --add tomcat
chkconfig --level 234 tomcat on
chkconfig --list tomcat
groupadd tomcat
useradd -g tomcat -d /home/tomcat -s /bin/sh tomcat
cd /etc/init.d/

chown tomcat:tomcat  tomcat

chmod +x tomcat
service tomcat start

check the log:
tail /application/tomcat/logs/catalina.out

Remove default content:

rm -rf /root/apache-tomcat-7.0.70.tar.gz
vim /tmp/run01.sh
#!/bin/bash
for i in {1..2}
 do
   for j in docs examples host-manager manager ROOT/RELEASE-NOTES.txt 
        do
                rm -rf /application/tomcat7_$i/webapps/$j
        done
 done

Change folder ownership and permissions:

for i in {1..2}; do chown -R tomcat.tomcat /application/tomcat7_$i; done
for i in {1..2}; do chmod g-w,o-rwx /application/tomcat7_$i; done

vim /tmp/run02.sh
#!/bin/bash
for i in {1..2}
 do
   for j in logs temp 
        do
                chmod o-rwx /application/tomcat7_$i/$j
        done
 done

vim /tmp/run03.sh
#!/bin/bash
for i in {1..2}
 do
   for j in bin webapps conf 
        do
                chmod g-w,o-rwx /application/tomcat7_$i/$j
        done
 done

for i in {1..2}; do chmod 770 /application/tomcat7_$i/conf/catalina.policy; done
vim /tmp/run04.sh
#!/bin/bash
for i in {1..2}
 do
   for j in catalina.properties context.xml logging.properties server.xml tomcat-users.xml web.xml
        do
                chmod g-w,o-rwx /application/tomcat7_$i/conf/$j
        done
 done

cd /application/tomcat/lib
mkdir -p org/apache/catalina/util/
jar xf catalina.jar org/apache/catalina/util/ServerInfo.properties
ll org/apache/catalina/util/ServerInfo.properties
vim org/apache/catalina/util/ServerInfo.properties
server.info=Secure Web server
server.number=1.0.0.0
server.built=Jan 01 2000 00:00:00 UTC
jar uf catalina.jar org/apache/catalina/util/ServerInfo.properties
rm -rf /application/tomcat/lib/org

optimize nginx performance

By Stan Zhou

Published 2016-10-17

1.优化worker进程个数

查看cpu总核数

1 2	# grep processor /proc/cpuinfo\|wc -l 2 #表示为1颗CPU四核

查看CPU总颗数：

1 2	# grep 'physical id' /proc/cpuinfo\|sort\|uniq\|wc -l 1 #对physical id去重计数，表示1颗cpu

# vim nginx.conf
worker_processes 2;
# /application/nginx/sbin/nginx -t
# /application/nginx/sbin/nginx -s reload

2.访问日志配置实战

sed -n '21,23 s/#//gp' ../conf/nginx.conf.default
log_format  main  '$remote_addr - $remote_user [$time_local] "$request" '
'$status $body_bytes_sent "$http_referer" '
'"$http_user_agent" "$http_x_forwarded_for"';

vim nginx.con
error_log  logs/error.log;
...
http {
...
log_format  main  '$remote_addr - $remote_user [$time_local] "$request" '
		  '$status $body_bytes_sent "$http_referer" '
		  '"$http_user_agent" "$http_x_forwarded_for"';

access_log  logs/access.log  main;
   
...
   }

Nginx访问日志轮询切割

# cat /server/script/cut_nginx_log.sh
#!/bin/sh
Dateformat=`date +%Y%m%d`
Basedir="/application/nginx"
Nginxlogdir="$Basedir/logs"
Logname="access"
[ -d $Nginxlogdir ] && cd $Nginxlogdir || exit 1
[ -f ${Logname}.log ]|| exit1
/bin/mv ${Logname}.log ${Dateformat}_${Logname}.log
$Basedir/sbin/nginx -s reload

cat >> /var/spool/cron/root <<EOF
#cut nginx access log by cwzhou
00 00 * * * /bin/sh /server/script/cut_nginx_log.sh > /dev/null 2>&1
EOF

#check result
#crontab -l

Nginx并发连接数量

Reference ngx_http_limit_conn_module
1.限制单ip并发连接数

vim nginx.conf
...
http{
    ...
    limit_conn_zone $binary_remote_addr zone=addr:10m;
    server{
	...
	location / {
		...
		limit_conn addr 1; #限制单ip的并发连接为1
        }
   }
}

1 2	ab -c 1 -n 10 http://10.0.0.3/ #模拟并发连接1,访问10次服务器 # -c为并发数 -n为请求书 10.0.0.3为nginx的Ip地址

应用场景之一是服务器下载：

1
2
3

location /download/ {
	limit_conn add 1;
}

2.限制虚拟主机总连接数

vim nginx.conf
...
http{
    ...
    limit_conn_zone $binary_remote_addr zone=addr:10m;
    limit_conn_zone $server_name zone=perserver:10m;

    server{
	...
	location / {
		...
		limit_conn perserver 2; #限制虚拟主机连接数为2
        }
   }
}

system admin tools

By Stan Zhou

Published 2016-10-13

system admin tool

iostat介绍
 全能系统监控工具dstat
most use:

1	[root@toad pxe]# dstat -cmsdnl -D sda3 -N lo,eth0 100 5

网络监控工具
本地服务器的网卡吞吐量：

1	iptraf -d eth0

# both
wget ftp://ftp.netperf.org/netperf/netperf-2.7.0.tar.gz
tar xf netperf-2.7.0.tar.gz
./configure
make
make install

#服务端：
[root@toad netperf-2.7.0]# netserver
Starting netserver with host 'IN(6)ADDR_ANY' port '12865' and family AF_UNSPEC
[root@toad netperf-2.7.0]# lsof -i :12865
COMMAND     PID USER   FD   TYPE  DEVICE SIZE/OFF NODE NAME
netserver 18230 root    3u  IPv6 2815424      0t0  TCP *:12865 (LISTEN)
[root@toad netperf-2.7.0]# ethtool eth0 |grep -i speed
        Speed: 1000Mb/s

#客户端
root@kali:~/Downloads/netperf-2.7.0#  netperf -H 192.168.199.231 -l 60
MIGRATED TCP STREAM TEST from 0.0.0.0 () port 0 AF_INET to 192.168.199.231 () port 0 AF_INET
Recv   Send    Send
Socket Socket  Message  Elapsed
Size   Size    Size     Time     Throughput
bytes  bytes   bytes    secs.    10^6bits/sec

87380  16384  16384    60.02      94.13
root@kali:~/Downloads/netperf-2.7.0# ethtool eth0 |grep -i speed
        Speed: 100Mb/s

Netperf缺省情况下进行TCP批量传输，即-t TCP_STREAM。测试过程中，netperf向netserver发送批量的TCP数据分组，以确定数据传输过程中的吞吐量：
从netperf的结果输出中，我们可以知道以下的一些信息：
1）远端系统（即server）使用大小为87380字节的socket接收缓冲
2）本地系统（即client）使用大小为16384字节的socket发送缓冲
3）向远端系统发送的测试分组大小为16384字节
4）测试经历的时间为60秒
5）吞吐量的测试结果为94.13Mbits/秒

Reference netperf 与网络性能测量

useful commands

Create a configuration file and add your license key \

echo “license_key: a61ac49b8930c041d9d940b8227f5d716819NRAL” | sudo tee -a /etc/newrelic-infra.yml && \

Create the agent’s yum repository \

sudo curl -o /etc/yum.repos.d/newrelic-infra.repo https://download.newrelic.com/infrastructure_agent/linux/yum/el/7/x86_64/newrelic-infra.repo && \

Update your yum cache \

sudo yum -q makecache -y –disablerepo=’*’ –enablerepo=’newrelic-infra’ && \

Run the installation script \

sudo yum install newrelic-infra -y

Written with StackEdit.

zabbix

By Stan Zhou

Published 2016-10-12

zabbix

IP address	Roles	notes
192.168.0.49(chatswood)	zabbix server	Centos 6.8 Zabbix 2.4.8
192.168.0.80(macadamina)	zabbix agent	Centos 6.8 Zabbix 2.4.8

server setup

yum install mysql-server -y
tar xf zabbix-2.4.8.tar.gz
cd zabbix-2.4.8
./configure --prefix=/opt/zabbix --enable-server --enable-agent --with-mysql
make && make install
yum install httpd php-devel php php-mbstring -y
vim /etc/hosts
192.168.0.80    macadamina
192.168.0.49    chatswood
cd /opt/zabbix/
mv etc{,_bak}
/bin/cp -rf etc_bak/ etc
[root@chatswood ~]# cat /opt/zabbix/etc/zabbix_server.conf|egrep -v "^#|^$"
LogFile=/tmp/zabbix_server.log
PidFile=/tmp/zabbix_server.pid
DBSocket=/tmp/mysql.sock
DBHost=chatswood
DBName=zabbix
DBUser=zabbix
DBPassword=123456
Timeout=30
[root@chatswood ~]# cat /opt/zabbix/etc/zabbix_agentd.conf|egrep -v "^#|^$"
PidFile=/tmp/zabbix_agentd.pid
LogFile=/tmp/zabbix_agentd.log
Server=chatswood
ServerActive=chatswood
Hostname=chatswood
Timeout=30

/etc/init.d/mysqld start
# mysql
mysql> create database zabbix character set utf8;
mysql> grant all on zabbix.* to zabbix@'192.168.0.49' identified by '123456';
mysql> flush privileges;
mysql> use zabbix;
mysql> show databases;
mysql> exit;
cd zabbix-2.4.8/database/mysql
mysql zabbix < ./schema.sql -p
mysql zabbix < ./images.sql -p
mysql zabbix < ./data.sql -p
## test database
mysql -u zabbix -p -h192.168.0.49
mysql> use zabbix;
mysql> show databases;
mysql> exit;

# create a zabbix user
groupadd -g 508 zabbix
useradd -g 508 -u 508 zabbix -M -s /sbin/nologin

/opt/zabbix/sbin/zabbix_server -c /opt/zabbix/etc/zabbix_server.conf
ps aux|grep zabbix
/opt/zabbix/sbin/zabbix_agentd -c /opt/zabbix/etc/zabbix_agentd.conf

# test
/opt/zabbix/bin/zabbix_get -s chatswood -k proc.num[]

cd zabbix-2.4.8
mkdir /var/www/html/zabbix
/bin/cp -rf frontends/php/* /var/www/html/zabbix/
vim /etc/php.ini
date.timezone = Asia/Shanghai
post_max_size = 18M
max_execution_time = 350
max_input_time = 350

yum install php-gd php-xml php-mysql php-bcmath -y
/etc/init.d/httpd restart
ps aux|grep apache
/usr/sbin/ntpdate 3.cn.pool.ntp.org
chown -R apache /var/www/html/zabbix/

agent setup

vim /etc/hosts
192.168.0.80    macadamina
192.168.0.49    chatswood
./configure --prefix=/opt/zabbix --enable-agent --with-net-snmp
make && make install
groupadd -g 508 zabbix
useradd -g 508 -u 508 zabbix -M -s /sbin/nologin
cat /opt/zabbix/etc/zabbix_agentd.conf|egrep -v "^#|^$"
LogFile=/tmp/zabbix_agentd.log
Server=chatswood
ServerActive=chatswood
Hostname=macadamina

/opt/zabbix/sbin/zabbix_agentd -c /opt/zabbix/etc/zabbix_agentd.conf
ps aux|grep zabbix
lsof|grep zabbix

Access http://192.168.0.49/zabbix,
default username: admin
password: zabbix

issue

checking for mysql_config… /usr/bin/mysql_config
checking for main in -lmysqlclient… no
configure: error: Not found mysqlclient library

#yum -y install mysql-devel

Written with StackEdit.

Amazon Linux 2

List Docker CLI commands

Beginning Devops with Docker

Chapter 1. Images and Containers

Docker-compose

Kubernetes

Docker ps command

Docker networking

upload that new image to dockerhub

Amazon web services and docker development

Working with ecs throught the aws cli

Docker registry

Error

Container Security

Containers inside VMs

Container Logging and Monitoring

Docker + EC2 Variations

ECS managed policies

virtual machine and Docker container difference

Commit Pipeline

Code Quality Stages

Triggers and notifications

Team development strategies

Kubernetes on the GKE

How do master nodes communicate?

Where can we run Kubernetes?

Can K8s be used in a hybird, multi-cloud world?

Interacting with K8s

Names and UIDs

Namespaces

Labels

Volumes

Lab: Volumes and the exmptydir volume

Persistent Volumes

Lab: use of secrets pass information to pods

Debugging

Insufficient cpu

Issue 1.

Solve:

Install Nginx

Install Nginx Package

Install Nginx

tomcat deploy on centos

软件准备

JDK下载：

Tomcat下载:

部署java环境

Install tomcat

Tomcat目录介绍

Tomcat多实例及集群架构

Tomcat多实例

jvm调优

service startup script

Remove default content:

Change folder ownership and permissions:

Remove Server Banner

1.优化worker进程个数

2.访问日志配置实战

Nginx并发连接数量

system admin tool

useful commands

Create a configuration file and add your license key \

Create the agent’s yum repository \

Update your yum cache \

Run the installation script \

zabbix

server setup

agent setup

issue