BogoToBogo
  • Home
  • About
  • Big Data
  • Machine Learning
  • AngularJS
  • Python
  • C++
  • go
  • DevOps
  • Kubernetes
  • Algorithms
  • More...
    • Qt 5
    • Linux
    • FFmpeg
    • Matlab
    • Django 1.8
    • Ruby On Rails
    • HTML5 & CSS

Docker Q & A

Docker_Icon.png Kubernetes-Icon.png




Bookmark and Share





bogotobogo.com site search:






Docker Q & A

  1. Difference between Docker Image and container.

    If a Docker application works on our local computer, it'll work anywhere that supports Docker. It greatly simplifies development process and can be a powerful tool for continuous delivery.

    To understand Docker, we need to know two key facets of how Docker works. Docker image vs container!

    Docker image is a kind of snapshot. We can think of it as a picture of a Docker virtual machine and the container is the virtual machine.

    A container is an instance of a docker image. In other words, a running instance of an image is a container. We can see all our images with docker images:

    $ docker images
    REPOSITORY          TAG                 IMAGE ID            CREATED             SIZE
    mysql               latest              8457e9155715        9 days ago          546MB
    busybox             latest              491198851f0c        2 weeks ago         1.23MB
    ubuntu              18.04               c090eaba6b94        6 weeks ago         63.3MB        
        

    To see our running containers with docker ps:

    $ docker ps
    CONTAINER ID        IMAGE               COMMAND             CREATED             STATUS              PORTS               NAMES
    2666137d9726        busybox             "sh"                2 hours ago         Up 2 hours                              tender_jones
    55ae35026d6c        ubuntu:18.04        "/bin/bash"         6 days ago          Up 6 days                               laughing_mcclintock        
        

    To see all containers including the ones stopped, we can use docker ps -a:

    $ docker ps -a
    CONTAINER ID        IMAGE               COMMAND                  CREATED              STATUS                     PORTS               NAMES
    5f6afe49bbd0        mysql               "docker-entrypoint.s…"   About a minute ago   Exited (0) 7 seconds ago                       angry_jang
    2666137d9726        busybox             "sh"                     2 hours ago          Up 2 hours                                     tender_jones
    55ae35026d6c        ubuntu:18.04        "/bin/bash"              6 days ago           Up 6 days                                      laughing_mcclintock        
        


  2. What is a Dockerfile?

    In order to build the application, we need to use a Dockerfile.

    In cooking, we need a recipe to make cookies. In Docker, we need a Dockerfile to build an image.

    A Dockerfile is simply a text-based script of instructions that is used to create a container image. Basically, it contains all the possible commands that a user may call on the command line to create an image.

    To see how it works, we'll use https://github.com/docker/getting-started.

    $ git clone https://github.com/docker/getting-started/tree/master/app  
    
    $ tree ../getting-started -L 2
    ../getting-started
    ├── Jenkinsfile
    ├── LICENSE
    ├── README.md
    ├── app
    │   ├── package.json
    │   ├── spec
    │   ├── src
    │   └── yarn.lock
    ├── build.sh
    ├── docker-compose.yml
    ├── docs
    │   ├── css
    │   ├── fonts
    │   ├── images
    │   ├── index.md
    │   └── tutorial
    ├── mkdocs.yml
    ├── requirements.txt
    └── yarn.lock
    

    1. Create a file named Dockerfile in the same folder as the file package.json with the following contents:

      FROM node:12-alpine
      WORKDIR /app
      COPY . .
      RUN yarn install --production
      CMD ["node", "src/index.js"]    
      

    2. Go to the app directory with the Dockerfile and build the container image using the docker build command:

      ~/getting-started/app $ ls
      Dockerfile	package.json	spec		src		yarn.lock
      
      ~/getting-started/app $ docker build -t getting-started .   
      
      $ docker images
      REPOSITORY          TAG                 IMAGE ID            CREATED             SIZE
      getting-started     latest              295a1b181e50        22 minutes ago      179MB
      

      Here, we instructed the builder that we wanted to start from the node:12-alpine image. The WORKDIR command is used to define the working directory of a Docker container at any given time. Any RUN, CMD, ADD, COPY, or ENTRYPOINT command will be executed in the specified working directory.

      After the image was downloaded, we copied in our application and used yarn to install our application's dependencies. The CMD specifies the default command to run when starting a container from this image.

      The -t flag tags our image to give it a human-readable name. Since we named the image getting-started, we can refer to that image when we run a container.

      The . at the end of the docker build command tells that Docker should look for the Dockerfile in the current directory.

      Now that we have an image, it's time to run the application using docker run command:

      $ docker run -d -p 3000:3000 getting-started    
      57198610f11c45a9eb4b7fd64fd93eabc2e0cbae17f980a8492618cb9822ba3f
      

      Note that we're running the new container in "detached" mode (in the background) and creating a mapping between the host's port 3000 to the container's port 3000. Without the port mapping, we wouldn't be able to access the application.


    3. Open a web browser to http://localhost:3000. We should see our app:

      node-app-port-3000.png

    4. One more thing about the CMD instruction: the difference between the CMD and ENTRYPOINT with related to the supplied to the docker run command. While the CMD will be completely over-written by the supplied command (or args), for the ENTRYPOINT, the supplied command will be appended to it (Dockerfile - Build Docker images automatically V - WORKDIR, ENV, ADD, and ENTRYPOINT).

      The Dockfile can be re-written as the following:

      FROM node:12-alpine
      WORKDIR /app
      COPY . .
      RUN yarn install --production
      ENTRYPOINT ["node"]
      CMD ["src/index.js"]                   
                 

      With the new Dockerfile, we can pass an arg to our docker run command, for example:

      $ docker run -d -p 3000:3000 getting-started src/index.js
      c85168de055f81f4390693c8029699266e69d304cdaf10e3c1c5a60d37040739               
                 

      Here, though we used the same arg, as in CMD ["src/index.js"], we could overwrite the arg provided by the CMD instruction.


    5. For more about Dockerfile instructions, checkout the following list:

      1. Dockerfile - Build Docker images automatically I - FROM, MAINTAINER, and build context
      2. Dockerfile - Build Docker images automatically II - revisiting FROM, MAINTAINER, build context, and caching
      3. Dockerfile - Build Docker images automatically III - RUN
      4. Dockerfile - Build Docker images automatically IV - CMD
      5. Dockerfile - Build Docker images automatically V - WORKDIR, ENV, ADD, and ENTRYPOINT


  3. Preferred way of removing containers?

    Let's stop the "busybox" container. First we need to do docker stop:

    $ docker stop 2666137d9726
    2666137d9726
        

    and then followed by a docker rm:
    $ docker rm 2666137d9726
    2666137d9726
        


  4. Difference between Docker pause and stop.

    $ docker run -it -d --name=busybox1 busybox /bin/sh
    007d5db147718b9fe6e2d2d4054fc6e1683836a74c4df11d08458e7d2a7e7018
    
    $ docker run -it -d --name=busybox2 busybox /bin/sh
    d00834397e505f86979ede9c9bd97bc2d4945d3f0c8f8486721a04165661ec68
    
    $ docker ps
    CONTAINER ID        IMAGE               COMMAND             CREATED             STATUS              PORTS               NAMES
    d00834397e50        busybox             "/bin/sh"           4 seconds ago       Up 4 seconds                            busybox2
    007d5db14771        busybox             "/bin/sh"           13 seconds ago      Up 13 seconds                           busybox1
        

    docker-lifecycle.png

    Picture credit: Get Started with Docker Lifecycle

    We can stop the "busybox1" and pause the "busybox2":

    $ docker stop busybox1
    busybox1
    
    $ docker pause busybox2
    busybox2
    
    $ docker ps
    CONTAINER ID        IMAGE               COMMAND             CREATED             STATUS                   PORTS               NAMES
    21f6d14521b8        busybox             "/bin/sh"           22 minutes ago      Up 45 seconds (Paused)                       busybox2
    

    The docker pause command suspends (via SIGSTOP signal) all processes in the specified containers.

    The docker stop command. The main process inside the container will receive SIGTERM, and after a grace period, SIGKILL.

    We cannot remove a paused container.

    $ docker rm busybox2
    Error response from daemon: You cannot remove a paused container 21f6d14521b8c16455166536000634e66097c7915f7608718e773e58e4e7aacf. 
    Unpause and then stop the container before attempting removal or force remove
    

    SIGSTOP is the pause signal that cannot be caught or ignored. The shell uses pausing (and its counterpart, resuming via SIGCONT) to implement job control.

    $ docker restart busybox2
    busybox2
    
    $ docker ps
    CONTAINER ID        IMAGE               COMMAND             CREATED             STATUS              PORTS               NAMES
    21f6d14521b8        busybox             "/bin/sh"           About an hour ago   Up 5 seconds                            busybox2
    55ae35026d6c        ubuntu:18.04        "/bin/bash"         6 days ago          Up 6 days                               laughing_mcclintock
    


  5. How can we create a Docker container in the Stopped state?
    $ docker create --name MyContainer ubuntu
    692b7ce25a7743907be877e7a758fd6a16b390a09275d73ade607dd25c4b0ee9
    

    We can see that it has created a new container. But we won’t see MyContainer because though it was created, it was never started.

    $ docker ps -a
    CONTAINER ID        IMAGE               COMMAND                  CREATED             STATUS                     PORTS               NAMES
    692b7ce25a77        ubuntu              "/bin/bash"              2 minutes ago       Created                                        MyContainer
    

    We can start this container with docker start command:

    $ docker start MyContainer
    MyContainer
    


  6. docker stats:

    The docker stats command returns a live data stream for running containers.

    $ docker ps
    CONTAINER ID        IMAGE               COMMAND               CREATED             STATUS              PORTS                    NAMES
    c85168de055f        getting-started     "node src/index.js"   2 hours ago         Up 2 hours          0.0.0.0:3000->3000/tcp   modest_easley
    55ae35026d6c        ubuntu:18.04        "/bin/bash"           8 days ago          Up 8 days                                    laughing_mcclintock
    
    $ docker stats
    CONTAINER ID        NAME                  CPU %               MEM USAGE / LIMIT     MEM %               NET I/O             BLOCK I/O           PIDS
    c85168de055f        modest_easley         0.00%               16.38MiB / 2.434GiB   0.66%               11.2kB / 5.41kB     0B / 0B             11
    55ae35026d6c        laughing_mcclintock   0.00%               1.125MiB / 2.434GiB   0.05%               14.9kB / 0B         0B / 0B             1    
    
    
    $ docker stats c85168de055f
    CONTAINER ID        NAME                CPU %               MEM USAGE / LIMIT     MEM %               NET I/O             BLOCK I/O           PIDS
    c85168de055f        modest_easley       0.00%               16.38MiB / 2.434GiB   0.66%               11.3kB / 5.41kB     0B / 0B             11
    


  7. docker system prune:

    It's a command used to remove all stopped containers, unused networks, build caches, and dangling images. The prune is one of the most useful commands in Docker:

    $ docker system prune   
    WARNING! This will remove:
      - all stopped containers
      - all networks not used by at least one container
      - all dangling images
      - all dangling build cache
    
    Are you sure you want to continue? [y/N] y
    Deleted Containers:
    1470e09b748c5a89a41a415d4bdfbeec61e4091d38eaceef94c96ac9edb90469
    692b7ce25a7743907be877e7a758fd6a16b390a09275d73ade607dd25c4b0ee9
    ...
    
    Deleted Images:
    deleted: sha256:295a1b181e50c37a3a9595bd498b5c980a9f90823473abdb8704ce3308628eef
    deleted: sha256:77234e845dbed0075aecab391e14bb9f1a34ec7bd34b88284d2f74b31b9837b0
    ...
    
    Total reclaimed space: 89.85MB
    


  8. docker-compose:

    Most of the time, we will most likely want to bring up all of the services listed in our docker-compose.yml and have the containers run their default command, so we would want to use docker-compose up.

    The docker-compose run command will spin up a new container for us to use while the docker-compose exec command will allow us to use a container that is already running.



  9. Multistage Image Builds:

    While using containers to build applications can be useful, it is important to distinguish between the build image and the runtime image.

    The build image contains all the tooling and libraries that are necessary to compile the application, while the runtime image contains the application to be deployed. A Java application has a build image that contains the JDK, Gradle/Maven, and compilation and testing tooling. Then our runtime image can contain only the Java runtime and our application.

    Compiling code as part of the image build is the most common ways of accidentally building large images.

    To resolve this issue, Docker introduced multistage builds. Rather than producing a single image, with the multistage builds, a Docker file can actually produce multiple images where each image is considered a stage. Artifacts can be copied from preceding stages to the current stage.

    The following two Dockerfiles demonstrate the multistage image builds from Use multi-stage builds:

    Dockerfile with number:
    FROM golang:1.7.3
    WORKDIR /go/src/github.com/alexellis/href-counter/
    RUN go get -d -v golang.org/x/net/html  
    COPY app.go .
    RUN CGO_ENABLED=0 GOOS=linux go build -a -installsuffix cgo -o app .
    
    FROM alpine:latest  
    RUN apk --no-cache add ca-certificates
    WORKDIR /root/
    COPY --from=0 /go/src/github.com/alexellis/href-counter/app .
    CMD ["./app"]      
    

    The second FROM instruction starts a new build stage with the alpine:latest image as its base. The COPY --from=0 line copies just the built artifact from the previous stage into this new stage. The Go SDK and any intermediate artifacts are left behind, and not saved in the final image.



    Dockerfile with name:
    FROM golang:1.7.3 AS builder
    WORKDIR /go/src/github.com/alexellis/href-counter/
    RUN go get -d -v golang.org/x/net/html  
    COPY app.go    .
    RUN CGO_ENABLED=0 GOOS=linux go build -a -installsuffix cgo -o app .
    
    FROM alpine:latest  
    RUN apk --no-cache add ca-certificates
    WORKDIR /root/
    COPY --from=builder /go/src/github.com/alexellis/href-counter/app .
    CMD ["./app"]   
    

    By default, the stages are not named, and we refer to them by their integer number, starting with 0 for the first FROM instruction as shown in the first Dockerfile. However, we can name our stages, by adding an AS <NAME> to the FROM instruction. The 2nd example improves the 1st one by naming the stages and using the name in the COPY instruction.



    Here is another sample that builds a "Go" application and runs the app.

    hello.go:

    package main
      
    import (
            "fmt"
            "log"
            "net/http"
    )
    
    //Hello Server responds to requests with the given URL path.
    func HelloServer(w http.ResponseWriter, r *http.Request) {
            fmt.Fprintf(w, "Hello, you requested: %s", r.URL.Path)
            log.Printf("Received request for path: %s", r.URL.Path)
    }
    func main() {
            var addr string = ":8181"
            handler := http.HandlerFunc(HelloServer)
            if err := http.ListenAndServe(addr, handler); err != nil {
                    log.Fatalf("Could not listen on port %s %v", addr, err)
            }
    }    
    

    Dockerfile that builds the app then copy the binary into a container:

    FROM golang:1-alpine as build
    WORKDIR /app
    COPY hello.go /app
    RUN go build hello.go
    
    FROM alpine:latest
    WORKDIR /app
    COPY --from=build /app /app
    EXPOSE 8180
    ENTRYPOINT ["./hello"]    
    

    Now, we have the following files in our working directory:

    $ ls
    Dockerfile	hello.go    
    

    We are now ready to build the image from the Dockerfile:

    $ docker build -t hello-go .
    Sending build context to Docker daemon  3.072kB
    Step 1/9 : FROM golang:1-alpine as build
     ---> 14ee78639386
    Step 2/9 : WORKDIR /app
     ---> Using cache
     ---> 295af3c2ffa0
    Step 3/9 : COPY hello.go /app
     ---> Using cache
     ---> debfbdb3c01d
    Step 4/9 : RUN go build hello.go
     ---> Using cache
     ---> e29f2ba09000
    Step 5/9 : FROM alpine:latest
     ---> 49f356fa4513
    Step 6/9 : WORKDIR /app
     ---> Running in 034e0cf20138
    Removing intermediate container 034e0cf20138
     ---> 03048fdf20d8
    Step 7/9 : COPY --from=build /app /app
     ---> f496e0e4bbcb
    Step 8/9 : EXPOSE 8180
     ---> Running in 7f2b631302e1
    Removing intermediate container 7f2b631302e1
     ---> c1545d176662
    Step 9/9 : ENTRYPOINT ["./hello"]
     ---> Running in dfeb253925e1
    Removing intermediate container dfeb253925e1
     ---> 10456b843247
    Successfully built 10456b843247
    Successfully tagged hello-go:latest  
    
    $ docker images
    REPOSITORY                    TAG                 IMAGE ID            CREATED              SIZE
    hello-go                      latest              10456b843247        About a minute ago   11.8MB
    

    To run the container and expose the internal port 8181 to our host port 8182:

    $ docker run -d --name hello-go-container --rm -p 8182:8181 hello-go
    907ad5290aa7a2db70496082906d25a291a5077933d343f6a8a7d50efa0760c7
    
    $ docker ps
    CONTAINER ID        IMAGE       COMMAND    CREATED             STATUS              PORTS                                NAMES
    907ad5290aa7        hello-go    "./hello"  16 seconds ago      Up 15 seconds       8180/tcp, 0.0.0.0:8182->8181/tcp     hello-go-container
    

    Test the application:

    $ curl localhost:8182
    Hello, you requested: /    
    

    It appears to be working fine!

    One more step. Let's go into the container and check the /app folder:

    $ docker exec -it hello-go-container /bin/sh
    /app # ls -la
    total 6044
    drwxr-xr-x    1 root     root          4096 Apr  6 23:47 .
    drwxr-xr-x    1 root     root          4096 Apr  6 23:53 ..
    -rwxr-xr-x    1 root     root       6176661 Apr  6 21:23 hello
    -rw-r--r--    1 root     root           493 Apr  6 20:22 hello.go    
    

    As we can see the binary (hello) has been copied successfully from build stage to the last stage.



  10. Sometimes when we run a docker container it exits immediately. Why?

    The short answer is that the container exits because it has no process to run.

    In this section, we'll also learn the difference between CMD and ENTRYPOINT

    When we run an Ubuntu image, it exits immediately as we can see below:

    $ docker run ubuntu:18.04
    Unable to find image 'ubuntu:18.04' locally
    18.04: Pulling from library/ubuntu
    6cf436f81810: Pull complete 
    987088a85b96: Pull complete 
    b4624b3efe06: Pull complete 
    d42beb8ded59: Pull complete 
    Digest: sha256:7a47ccc3bbe8a451b500d2b53104868b46d60ee8f5b35a24b41a86077c650210
    Status: Downloaded newer image for ubuntu:18.04
    
    $ docker ps -a
    CONTAINER ID IMAGE        COMMAND     CREATED        STATUS                    PORTS                                              NAMES
    9ba1aa158caf ubuntu:18.04 "/bin/bash" 11 seconds ago Exited (0) 10 seconds ago                                                    youthful_stonebraker
    


    Why is that? Why it exited?

    Unlike VMs which are meant to host OS, containers are meant to run a task or a process such as a web server/application or a db. So, once a task is complete, a container exits. A container lives as long as a process within it is running. If an application in a container crashes, container exits.

    So, who defines which process should be running inside a container?

    Let's look into the following Dockerfile for nginx, specially the CMD[] instruction:

    #
    # Nginx Dockerfile
    #
    # https://github.com/dockerfile/nginx
    #
    
    # Pull base image.
    FROM dockerfile/ubuntu
    
    # Install Nginx.
    RUN \
      add-apt-repository -y ppa:nginx/stable && \
      apt-get update && \
      apt-get install -y nginx && \
      rm -rf /var/lib/apt/lists/* && \
      echo "\ndaemon off;" >> /etc/nginx/nginx.conf && \
      chown -R www-data:www-data /var/lib/nginx
    
    # Define mountable directories.
    VOLUME ["/etc/nginx/sites-enabled", "/etc/nginx/certs", "/etc/nginx/conf.d", "/var/log/nginx", "/var/www/html"]
    
    # Define working directory.
    WORKDIR /etc/nginx
    
    # Define default command.
    CMD ["nginx"]
    
    # Expose ports.
    EXPOSE 80
    EXPOSE 443
    

    Yes, the CMD[] tells the Docker which program should be run when the container starts. In our case, it is the "nginx" command.


    For mysql Dockerfile it is mysqld command:

    COPY docker-entrypoint.sh /entrypoint.sh
    COPY healthcheck.sh /healthcheck.sh
    ENTRYPOINT ["/entrypoint.sh"]
    HEALTHCHECK CMD /healthcheck.sh
    EXPOSE 3306 33060
    CMD ["mysqld"]
    

    How about our Ubuntu image Dockerfile?

    ...
    CMD ["/bin/bash"]
    

    It uses bash for its default command.

    Unlike the web server or a db, the bash is not a process, it's just a shell listening and waiting for an input. If it does not get any from a terminal, it exits.

    Earlier, when we run a container from the Ubuntu image, it launches a "bash" program but the Docker, by default, not attaching any terminal to a container when it runs. So, the container could not find a terminal, and just exited.

    We can make container alive for a while by overwriting the CMD ["/bin/bash"], for example, sleep 30s when we run docker:

    $ docker run ubuntu:18.04 sleep 30s
    
    $ docker ps
    CONTAINER ID  IMAGE          COMMAND       CREATED         STATUS        PORTS  NAMES
    55ab52fa884d  ubuntu:18.04   "sleep 30s"   7 seconds ago   Up 6 seconds         relaxed_euler
    

    But how we can make the container always run the sleep command when it starts? Note that we added it to the docker run command.

    One way to avoid adding the "sleep 30s" after the command is to use the CMD instruction in our Dockerfile:

    FROM ubuntu:18.04
    CMD sleep 30
    

    Or we can use array:

    FROM ubuntu:18.04
    CMD ["sleep", "30"]
    

    Note that we should NOT use the following because the command and args should be separated:

    CMD ["sleep 30"] X
    

    Now we can build our image with a name of "ubuntu-sleep":

    $ docker build -t ubuntu-sleep .
    Sending build context to Docker daemon  2.048kB
    Step 1/2 : FROM ubuntu:18.04
     ---> 47b19964fb50
    Step 2/2 : CMD ["sleep", "30"]
     ---> Running in c84ecc7a5b3d
    Removing intermediate container c84ecc7a5b3d
     ---> 3f21ee94c150
    Successfully built 3f21ee94c150
    Successfully tagged ubuntu-sleep:latest
    

    Then, run a container from the newly created image:

    $ docker run ubuntu-sleep
    

    The container always sleeps 30s after it started!

    But we have a problem. What if we want to change the sleep time?

    Currently, it's been hard-coded.

    Of course, we can overwrite the command like this:

    $ docker run ubuntu-sleep sleep 5
    

    However, because the image name itself is already indicating it would sleep, we need to find a way of just feeding the seconds as an argument not with the sleep command, and the image automatically invoke the "sleep" command needing only the parameter. Something like this:

    $ docker run ubuntu-sleep 5
    

    That's why we need the ENTRYPOINT instruction.

    It simply specifies a program to run when a container starts.

    So, our Dockerfile should be changed from:

    FROM ubuntu:18.04
    CMD ["sleep", "30"]
    

    to:

    FROM ubuntu:18.04
    ENTRYPOINT ["sleep"]
    

    Build a new image and run the container:

    $ docker build -t ubuntu-sleep .
    Sending build context to Docker daemon  2.048kB
    Step 1/2 : FROM ubuntu:18.04
     ---> 47b19964fb50
    Step 2/2 : ENTRYPOINT ["sleep"]
     ---> Running in e5e6e83e9e01
    Removing intermediate container e5e6e83e9e01
     ---> affbc2e6ed86
    Successfully built affbc2e6ed86
    Successfully tagged ubuntu-sleep:latest
    
    $ docker run ubuntu-sleep 5
    

    Note the difference between the CMD and ENTRYPOINT with related to the supplied to the docker run command. While the CMD will be completely over-written by the supplied command (or args), for the ENTRYPOINT, the supplied command will be appended to it.

    Another problem in our Dockerfile: let's see:

    $ docker run ubuntu-sleep
    sleep: missing operand
    Try 'sleep --help' for more information.
    

    In the command above, we did not supply an arg for the sleep command, and got an error when the container started.

    We need a default value for the command so that container runs event though an arg is missing.

    Here is where the CMD comes into play: the CMD instruction will be appended to the ENTRYPOINT instruction.

    Here is our new Dockerfile:

    FROM ubuntu:18.04
    ENTRYPOINT ["sleep"]
    CMD ["5"]
    

    Build the image and run a container from the image, and we should not get any error when we do not specify sleep time:

    $ docker build -t ubuntu-sleep .
    $ docker run ubuntu-sleep
    

    If we add a parameter to the command, it will overwrites the default value specified in CMD.

    One more thing regarding the ENTRYPOINT. What if we want to override the command specified in the ENTRYPOINT?

    In that case, we can give a new command in docker run command, for example:

    $ docker run --entrypoint new-sleep-command ubuntu-sleep 60
    


    Let's go further and look into how the ENTRYPOINT and CMD in Dockerfile are translated in a Pod definition yaml file:

    pod-definition-yaml.png

    Picture source Docker for Beginners - Commands vs Entrypoint - Kubernetes

    As we can see the parameters in ENTRYPOINT and CMD can be overwritten with the ones provided via "command" are "args" in "spec.containers" of the yaml.




  11. How we can run a container with "root" privilege?

    $ docker exec -u 0 -it my-container sh    
    






  12. How to push an image to AWS ECR?

    Docker login first assuming the AWS credentials are in place either from ENV or from ~/.aws/credentials:

    $ aws --region us-west-2 ecr get-login-password \                                        
        | docker login \
            --password-stdin \
            --username AWS \
            437028470429.dkr.ecr.us-west-2.amazonaws.com
            
    Login Succeeded
    

  13. Then, push it to ECR:

    $ docker push 437028470429.dkr.ecr.us-west-2.amazonaws.com/test-khong:0.2.1
    The push refers to repository [437028470429.dkr.ecr.us-west-2.amazonaws.com/test-khong] 
    ...
    0.2.1: digest: sha256:ca2fc17f610...2ceaddf5e0cbda5e74 size: 4484
    








Docker & K8s

  1. Docker install on Amazon Linux AMI
  2. Docker install on EC2 Ubuntu 14.04
  3. Docker container vs Virtual Machine
  4. Docker install on Ubuntu 14.04
  5. Docker Hello World Application
  6. Nginx image - share/copy files, Dockerfile
  7. Working with Docker images : brief introduction
  8. Docker image and container via docker commands (search, pull, run, ps, restart, attach, and rm)
  9. More on docker run command (docker run -it, docker run --rm, etc.)
  10. Docker Networks - Bridge Driver Network
  11. Docker Persistent Storage
  12. File sharing between host and container (docker run -d -p -v)
  13. Linking containers and volume for datastore
  14. Dockerfile - Build Docker images automatically I - FROM, MAINTAINER, and build context
  15. Dockerfile - Build Docker images automatically II - revisiting FROM, MAINTAINER, build context, and caching
  16. Dockerfile - Build Docker images automatically III - RUN
  17. Dockerfile - Build Docker images automatically IV - CMD
  18. Dockerfile - Build Docker images automatically V - WORKDIR, ENV, ADD, and ENTRYPOINT
  19. Docker - Apache Tomcat
  20. Docker - NodeJS
  21. Docker - NodeJS with hostname
  22. Docker Compose - NodeJS with MongoDB
  23. Docker - Prometheus and Grafana with Docker-compose
  24. Docker - StatsD/Graphite/Grafana
  25. Docker - Deploying a Java EE JBoss/WildFly Application on AWS Elastic Beanstalk Using Docker Containers
  26. Docker : NodeJS with GCP Kubernetes Engine
  27. Docker : Jenkins Multibranch Pipeline with Jenkinsfile and Github
  28. Docker : Jenkins Master and Slave
  29. Docker - ELK : ElasticSearch, Logstash, and Kibana
  30. Docker - ELK 7.6 : Elasticsearch on Centos 7
  31. Docker - ELK 7.6 : Filebeat on Centos 7
  32. Docker - ELK 7.6 : Logstash on Centos 7
  33. Docker - ELK 7.6 : Kibana on Centos 7
  34. Docker - ELK 7.6 : Elastic Stack with Docker Compose
  35. Docker - Deploy Elastic Cloud on Kubernetes (ECK) via Elasticsearch operator on minikube
  36. Docker - Deploy Elastic Stack via Helm on minikube
  37. Docker Compose - A gentle introduction with WordPress
  38. Docker Compose - MySQL
  39. MEAN Stack app on Docker containers : micro services
  40. MEAN Stack app on Docker containers : micro services via docker-compose
  41. Docker Compose - Hashicorp's Vault and Consul Part A (install vault, unsealing, static secrets, and policies)
  42. Docker Compose - Hashicorp's Vault and Consul Part B (EaaS, dynamic secrets, leases, and revocation)
  43. Docker Compose - Hashicorp's Vault and Consul Part C (Consul)
  44. Docker Compose with two containers - Flask REST API service container and an Apache server container
  45. Docker compose : Nginx reverse proxy with multiple containers
  46. Docker & Kubernetes : Envoy - Getting started
  47. Docker & Kubernetes : Envoy - Front Proxy
  48. Docker & Kubernetes : Ambassador - Envoy API Gateway on Kubernetes
  49. Docker Packer
  50. Docker Cheat Sheet
  51. Docker Q & A #1
  52. Kubernetes Q & A - Part I
  53. Kubernetes Q & A - Part II
  54. Docker - Run a React app in a docker
  55. Docker - Run a React app in a docker II (snapshot app with nginx)
  56. Docker - NodeJS and MySQL app with React in a docker
  57. Docker - Step by Step NodeJS and MySQL app with React - I
  58. Installing LAMP via puppet on Docker
  59. Docker install via Puppet
  60. Nginx Docker install via Ansible
  61. Apache Hadoop CDH 5.8 Install with QuickStarts Docker
  62. Docker - Deploying Flask app to ECS
  63. Docker Compose - Deploying WordPress to AWS
  64. Docker - WordPress Deploy to ECS with Docker-Compose (ECS-CLI EC2 type)
  65. Docker - WordPress Deploy to ECS with Docker-Compose (ECS-CLI Fargate type)
  66. Docker - ECS Fargate
  67. Docker - AWS ECS service discovery with Flask and Redis
  68. Docker & Kubernetes : minikube
  69. Docker & Kubernetes 2 : minikube Django with Postgres - persistent volume
  70. Docker & Kubernetes 3 : minikube Django with Redis and Celery
  71. Docker & Kubernetes 4 : Django with RDS via AWS Kops
  72. Docker & Kubernetes : Kops on AWS
  73. Docker & Kubernetes : Ingress controller on AWS with Kops
  74. Docker & Kubernetes : HashiCorp's Vault and Consul on minikube
  75. Docker & Kubernetes : HashiCorp's Vault and Consul - Auto-unseal using Transit Secrets Engine
  76. Docker & Kubernetes : Persistent Volumes & Persistent Volumes Claims - hostPath and annotations
  77. Docker & Kubernetes : Persistent Volumes - Dynamic volume provisioning
  78. Docker & Kubernetes : DaemonSet
  79. Docker & Kubernetes : Secrets
  80. Docker & Kubernetes : kubectl command
  81. Docker & Kubernetes : Assign a Kubernetes Pod to a particular node in a Kubernetes cluster
  82. Docker & Kubernetes : Configure a Pod to Use a ConfigMap
  83. AWS : EKS (Elastic Container Service for Kubernetes)
  84. Docker & Kubernetes : Run a React app in a minikube
  85. Docker & Kubernetes : Minikube install on AWS EC2
  86. Docker & Kubernetes : Cassandra with a StatefulSet
  87. Docker & Kubernetes : Terraform and AWS EKS
  88. Docker & Kubernetes : Pods and Service definitions
  89. Docker & Kubernetes : Service IP and the Service Type
  90. Docker & Kubernetes : Kubernetes DNS with Pods and Services
  91. Docker & Kubernetes : Headless service and discovering pods
  92. Docker & Kubernetes : Scaling and Updating application
  93. Docker & Kubernetes : Horizontal pod autoscaler on minikubes
  94. Docker & Kubernetes : From a monolithic app to micro services on GCP Kubernetes
  95. Docker & Kubernetes : Rolling updates
  96. Docker & Kubernetes : Deployments to GKE (Rolling update, Canary and Blue-green deployments)
  97. Docker & Kubernetes : Slack Chat Bot with NodeJS on GCP Kubernetes
  98. Docker & Kubernetes : Continuous Delivery with Jenkins Multibranch Pipeline for Dev, Canary, and Production Environments on GCP Kubernetes
  99. Docker & Kubernetes : NodePort vs LoadBalancer vs Ingress
  100. Docker & Kubernetes : MongoDB / MongoExpress on Minikube
  101. Docker & Kubernetes : Load Testing with Locust on GCP Kubernetes
  102. Docker & Kubernetes : MongoDB with StatefulSets on GCP Kubernetes Engine
  103. Docker & Kubernetes : Nginx Ingress Controller on Minikube
  104. Docker & Kubernetes : Setting up Ingress with NGINX Controller on Minikube (Mac)
  105. Docker & Kubernetes : Nginx Ingress Controller for Dashboard service on Minikube
  106. Docker & Kubernetes : Nginx Ingress Controller on GCP Kubernetes
  107. Docker & Kubernetes : Kubernetes Ingress with AWS ALB Ingress Controller in EKS
  108. Docker & Kubernetes : Setting up a private cluster on GCP Kubernetes
  109. Docker & Kubernetes : Kubernetes Namespaces (default, kube-public, kube-system) and switching namespaces (kubens)
  110. Docker & Kubernetes : StatefulSets on minikube
  111. Docker & Kubernetes : RBAC
  112. Docker & Kubernetes Service Account, RBAC, and IAM
  113. Docker & Kubernetes - Kubernetes Service Account, RBAC, IAM with EKS ALB, Part 1
  114. Docker & Kubernetes : Helm Chart
  115. Docker & Kubernetes : My first Helm deploy
  116. Docker & Kubernetes : Readiness and Liveness Probes
  117. Docker & Kubernetes : Helm chart repository with Github pages
  118. Docker & Kubernetes : Deploying WordPress and MariaDB with Ingress to Minikube using Helm Chart
  119. Docker & Kubernetes : Deploying WordPress and MariaDB to AWS using Helm 2 Chart
  120. Docker & Kubernetes : Deploying WordPress and MariaDB to AWS using Helm 3 Chart
  121. Docker & Kubernetes : Helm Chart for Node/Express and MySQL with Ingress
  122. Docker & Kubernetes : Deploy Prometheus and Grafana using Helm and Prometheus Operator - Monitoring Kubernetes node resources out of the box
  123. Docker & Kubernetes : Deploy Prometheus and Grafana using kube-prometheus-stack Helm Chart
  124. Docker & Kubernetes : Istio (service mesh) sidecar proxy on GCP Kubernetes
  125. Docker & Kubernetes : Istio on EKS
  126. Docker & Kubernetes : Istio on Minikube with AWS EC2 for Bookinfo Application
  127. Docker & Kubernetes : Deploying .NET Core app to Kubernetes Engine and configuring its traffic managed by Istio (Part I)
  128. Docker & Kubernetes : Deploying .NET Core app to Kubernetes Engine and configuring its traffic managed by Istio (Part II - Prometheus, Grafana, pin a service, split traffic, and inject faults)
  129. Docker & Kubernetes : Helm Package Manager with MySQL on GCP Kubernetes Engine
  130. Docker & Kubernetes : Deploying Memcached on Kubernetes Engine
  131. Docker & Kubernetes : EKS Control Plane (API server) Metrics with Prometheus
  132. Docker & Kubernetes : Spinnaker on EKS with Halyard
  133. Docker & Kubernetes : Continuous Delivery Pipelines with Spinnaker and Kubernetes Engine
  134. Docker & Kubernetes : Multi-node Local Kubernetes cluster : Kubeadm-dind (docker-in-docker)
  135. Docker & Kubernetes : Multi-node Local Kubernetes cluster : Kubeadm-kind (k8s-in-docker)
  136. Docker & Kubernetes : nodeSelector, nodeAffinity, taints/tolerations, pod affinity and anti-affinity - Assigning Pods to Nodes
  137. Docker & Kubernetes : Jenkins-X on EKS
  138. Docker & Kubernetes : ArgoCD App of Apps with Heml on Kubernetes
  139. Docker & Kubernetes : ArgoCD on Kubernetes cluster
  140. Docker & Kubernetes : GitOps with ArgoCD for Continuous Delivery to Kubernetes clusters (minikube) - guestbook



Ph.D. / Golden Gate Ave, San Francisco / Seoul National Univ / Carnegie Mellon / UC Berkeley / DevOps / Deep Learning / Visualization





Docker & K8s



Docker install on Amazon Linux AMI

Docker install on EC2 Ubuntu 14.04

Docker container vs Virtual Machine

Docker install on Ubuntu 14.04

Docker Hello World Application

Nginx image - share/copy files, Dockerfile

Working with Docker images : brief introduction

Docker image and container via docker commands (search, pull, run, ps, restart, attach, and rm)

More on docker run command (docker run -it, docker run --rm, etc.)

Docker Networks - Bridge Driver Network

Docker Persistent Storage

File sharing between host and container (docker run -d -p -v)

Linking containers and volume for datastore

Dockerfile - Build Docker images automatically I - FROM, MAINTAINER, and build context

Dockerfile - Build Docker images automatically II - revisiting FROM, MAINTAINER, build context, and caching

Dockerfile - Build Docker images automatically III - RUN

Dockerfile - Build Docker images automatically IV - CMD

Dockerfile - Build Docker images automatically V - WORKDIR, ENV, ADD, and ENTRYPOINT

Docker - Apache Tomcat

Docker - NodeJS

Docker - NodeJS with hostname

Docker Compose - NodeJS with MongoDB

Docker - Prometheus and Grafana with Docker-compose

Docker - StatsD/Graphite/Grafana

Docker - Deploying a Java EE JBoss/WildFly Application on AWS Elastic Beanstalk Using Docker Containers

Docker : NodeJS with GCP Kubernetes Engine

Docker : Jenkins Multibranch Pipeline with Jenkinsfile and Github

Docker : Jenkins Master and Slave

Docker - ELK : ElasticSearch, Logstash, and Kibana

Docker - ELK 7.6 : Elasticsearch on Centos 7 Docker - ELK 7.6 : Filebeat on Centos 7

Docker - ELK 7.6 : Logstash on Centos 7

Docker - ELK 7.6 : Kibana on Centos 7 Part 1

Docker - ELK 7.6 : Kibana on Centos 7 Part 2

Docker - ELK 7.6 : Elastic Stack with Docker Compose

Docker - Deploy Elastic Cloud on Kubernetes (ECK) via Elasticsearch operator on minikube

Docker - Deploy Elastic Stack via Helm on minikube

Docker Compose - A gentle introduction with WordPress

Docker Compose - MySQL

MEAN Stack app on Docker containers : micro services

Docker Compose - Hashicorp's Vault and Consul Part A (install vault, unsealing, static secrets, and policies)

Docker Compose - Hashicorp's Vault and Consul Part B (EaaS, dynamic secrets, leases, and revocation)

Docker Compose - Hashicorp's Vault and Consul Part C (Consul)

Docker Compose with two containers - Flask REST API service container and an Apache server container

Docker compose : Nginx reverse proxy with multiple containers

Docker compose : Nginx reverse proxy with multiple containers

Docker & Kubernetes : Envoy - Getting started

Docker & Kubernetes : Envoy - Front Proxy

Docker & Kubernetes : Ambassador - Envoy API Gateway on Kubernetes

Docker Packer

Docker Cheat Sheet

Docker Q & A

Kubernetes Q & A - Part I

Kubernetes Q & A - Part II

Docker - Run a React app in a docker

Docker - Run a React app in a docker II (snapshot app with nginx)

Docker - NodeJS and MySQL app with React in a docker

Docker - Step by Step NodeJS and MySQL app with React - I

Installing LAMP via puppet on Docker

Docker install via Puppet

Nginx Docker install via Ansible

Apache Hadoop CDH 5.8 Install with QuickStarts Docker

Docker - Deploying Flask app to ECS

Docker Compose - Deploying WordPress to AWS

Docker - WordPress Deploy to ECS with Docker-Compose (ECS-CLI EC2 type)

Docker - ECS Fargate

Docker - AWS ECS service discovery with Flask and Redis

Docker & Kubernetes: minikube version: v1.31.2, 2023

Docker & Kubernetes 1 : minikube

Docker & Kubernetes 2 : minikube Django with Postgres - persistent volume

Docker & Kubernetes 3 : minikube Django with Redis and Celery

Docker & Kubernetes 4 : Django with RDS via AWS Kops

Docker & Kubernetes : Kops on AWS

Docker & Kubernetes : Ingress controller on AWS with Kops

Docker & Kubernetes : HashiCorp's Vault and Consul on minikube

Docker & Kubernetes : HashiCorp's Vault and Consul - Auto-unseal using Transit Secrets Engine

Docker & Kubernetes : Persistent Volumes & Persistent Volumes Claims - hostPath and annotations

Docker & Kubernetes : Persistent Volumes - Dynamic volume provisioning

Docker & Kubernetes : DaemonSet

Docker & Kubernetes : Secrets

Docker & Kubernetes : kubectl command

Docker & Kubernetes : Assign a Kubernetes Pod to a particular node in a Kubernetes cluster

Docker & Kubernetes : Configure a Pod to Use a ConfigMap

AWS : EKS (Elastic Container Service for Kubernetes)

Docker & Kubernetes : Run a React app in a minikube

Docker & Kubernetes : Minikube install on AWS EC2

Docker & Kubernetes : Cassandra with a StatefulSet

Docker & Kubernetes : Terraform and AWS EKS

Docker & Kubernetes : Pods and Service definitions

Docker & Kubernetes : Headless service and discovering pods

Docker & Kubernetes : Service IP and the Service Type

Docker & Kubernetes : Kubernetes DNS with Pods and Services

Docker & Kubernetes - Scaling and Updating application

Docker & Kubernetes : Horizontal pod autoscaler on minikubes

Docker & Kubernetes : NodePort vs LoadBalancer vs Ingress

Docker & Kubernetes : Load Testing with Locust on GCP Kubernetes

Docker & Kubernetes : From a monolithic app to micro services on GCP Kubernetes

Docker & Kubernetes : Rolling updates

Docker & Kubernetes : Deployments to GKE (Rolling update, Canary and Blue-green deployments)

Docker & Kubernetes : Slack Chat Bot with NodeJS on GCP Kubernetes

Docker & Kubernetes : Continuous Delivery with Jenkins Multibranch Pipeline for Dev, Canary, and Production Environments on GCP Kubernetes

Docker & Kubernetes - MongoDB with StatefulSets on GCP Kubernetes Engine

Docker & Kubernetes : Nginx Ingress Controller on minikube

Docker & Kubernetes : Setting up Ingress with NGINX Controller on Minikube (Mac)

Docker & Kubernetes : Nginx Ingress Controller for Dashboard service on Minikube

Docker & Kubernetes : Nginx Ingress Controller on GCP Kubernetes

Docker & Kubernetes : Kubernetes Ingress with AWS ALB Ingress Controller in EKS

Docker & Kubernetes : MongoDB / MongoExpress on Minikube

Docker & Kubernetes : Setting up a private cluster on GCP Kubernetes

Docker & Kubernetes : Kubernetes Namespaces (default, kube-public, kube-system) and switching namespaces (kubens)

Docker & Kubernetes : StatefulSets on minikube

Docker & Kubernetes : StatefulSets on minikube

Docker & Kubernetes : RBAC

Docker & Kubernetes Service Account, RBAC, and IAM

Docker & Kubernetes - Kubernetes Service Account, RBAC, IAM with EKS ALB, Part 1

Docker & Kubernetes : Helm Chart

Docker & Kubernetes : My first Helm deploy

Docker & Kubernetes : Readiness and Liveness Probes

Docker & Kubernetes : Helm chart repository with Github pages

Docker & Kubernetes : Deploying WordPress and MariaDB with Ingress to Minikube using Helm Chart

Docker & Kubernetes : Deploying WordPress and MariaDB to AWS using Helm 2 Chart

Docker & Kubernetes : Deploying WordPress and MariaDB to AWS using Helm 3 Chart

Docker & Kubernetes : Helm Chart for Node/Express and MySQL with Ingress

Docker & Kubernetes : Docker_Helm_Chart_Node_Expess_MySQL_Ingress.php

Docker & Kubernetes: Deploy Prometheus and Grafana using Helm and Prometheus Operator - Monitoring Kubernetes node resources out of the box

Docker & Kubernetes : Deploy Prometheus and Grafana using kube-prometheus-stack Helm Chart

Docker & Kubernetes : Istio (service mesh) sidecar proxy on GCP Kubernetes

Docker & Kubernetes : Istio on EKS

Docker & Kubernetes : Istio on Minikube with AWS EC2 for Bookinfo Application

Docker & Kubernetes : Deploying .NET Core app to Kubernetes Engine and configuring its traffic managed by Istio (Part I)

Docker & Kubernetes : Deploying .NET Core app to Kubernetes Engine and configuring its traffic managed by Istio (Part II - Prometheus, Grafana, pin a service, split traffic, and inject faults)

Docker & Kubernetes : Helm Package Manager with MySQL on GCP Kubernetes Engine

Docker & Kubernetes : Deploying Memcached on Kubernetes Engine

Docker & Kubernetes : EKS Control Plane (API server) Metrics with Prometheus

Docker & Kubernetes : Spinnaker on EKS with Halyard

Docker & Kubernetes : Continuous Delivery Pipelines with Spinnaker and Kubernetes Engine

Docker & Kubernetes: Multi-node Local Kubernetes cluster - Kubeadm-dind(docker-in-docker)

Docker & Kubernetes: Multi-node Local Kubernetes cluster - Kubeadm-kind(k8s-in-docker)

Docker & Kubernetes : nodeSelector, nodeAffinity, taints/tolerations, pod affinity and anti-affinity - Assigning Pods to Nodes

Docker & Kubernetes : Jenkins-X on EKS

Docker & Kubernetes : ArgoCD App of Apps with Heml on Kubernetes

Docker & Kubernetes : ArgoCD on Kubernetes cluster

Docker & Kubernetes : GitOps with ArgoCD for Continuous Delivery to Kubernetes clusters (minikube) - guestbook




Sponsor Open Source development activities and free contents for everyone.

Thank you.

- K Hong







Ansible 2.0



What is Ansible?

Quick Preview - Setting up web servers with Nginx, configure environments, and deploy an App

SSH connection & running commands

Ansible: Playbook for Tomcat 9 on Ubuntu 18.04 systemd with AWS

Modules

Playbooks

Handlers

Roles

Playbook for LAMP HAProxy

Installing Nginx on a Docker container

AWS : Creating an ec2 instance & adding keys to authorized_keys

AWS : Auto Scaling via AMI

AWS : creating an ELB & registers an EC2 instance from the ELB

Deploying Wordpress micro-services with Docker containers on Vagrant box via Ansible

Setting up Apache web server

Deploying a Go app to Minikube

Ansible with Terraform





Terraform



Introduction to Terraform with AWS elb & nginx

Terraform Tutorial - terraform format(tf) and interpolation(variables)

Terraform Tutorial - user_data

Terraform Tutorial - variables

Terraform 12 Tutorial - Loops with count, for_each, and for

Terraform Tutorial - creating multiple instances (count, list type and element() function)

Terraform Tutorial - State (terraform.tfstate) & terraform import

Terraform Tutorial - Output variables

Terraform Tutorial - Destroy

Terraform Tutorial - Modules

Terraform Tutorial - Creating AWS S3 bucket / SQS queue resources and notifying bucket event to queue

Terraform Tutorial - AWS ASG and Modules

Terraform Tutorial - VPC, Subnets, RouteTable, ELB, Security Group, and Apache server I

Terraform Tutorial - VPC, Subnets, RouteTable, ELB, Security Group, and Apache server II

Terraform Tutorial - Docker nginx container with ALB and dynamic autoscaling

Terraform Tutorial - AWS ECS using Fargate : Part I

Hashicorp Vault

HashiCorp Vault Agent

HashiCorp Vault and Consul on AWS with Terraform

Ansible with Terraform

AWS IAM user, group, role, and policies - part 1

AWS IAM user, group, role, and policies - part 2

Delegate Access Across AWS Accounts Using IAM Roles

AWS KMS

terraform import & terraformer import

Terraform commands cheat sheet

Terraform Cloud

Terraform 14

Creating Private TLS Certs





DevOps



Phases of Continuous Integration

Software development methodology

Introduction to DevOps

Samples of Continuous Integration (CI) / Continuous Delivery (CD) - Use cases

Artifact repository and repository management

Linux - General, shell programming, processes & signals ...

RabbitMQ...

MariaDB

New Relic APM with NodeJS : simple agent setup on AWS instance

Nagios on CentOS 7 with Nagios Remote Plugin Executor (NRPE)

Nagios - The industry standard in IT infrastructure monitoring on Ubuntu

Zabbix 3 install on Ubuntu 14.04 & adding hosts / items / graphs

Datadog - Monitoring with PagerDuty/HipChat and APM

Install and Configure Mesos Cluster

Cassandra on a Single-Node Cluster

Container Orchestration : Docker Swarm vs Kubernetes vs Apache Mesos

OpenStack install on Ubuntu 16.04 server - DevStack

AWS EC2 Container Service (ECS) & EC2 Container Registry (ECR) | Docker Registry

CI/CD with CircleCI - Heroku deploy

Introduction to Terraform with AWS elb & nginx

Docker & Kubernetes

Kubernetes I - Running Kubernetes Locally via Minikube

Kubernetes II - kops on AWS

Kubernetes III - kubeadm on AWS

AWS : EKS (Elastic Container Service for Kubernetes)

CI/CD Github actions

CI/CD Gitlab



DevOps / Sys Admin Q & A



(1A) - Linux Commands

(1B) - Linux Commands

(2) - Networks

(2B) - Networks

(3) - Linux Systems

(4) - Scripting (Ruby/Shell)

(5) - Configuration Management

(6) - AWS VPC setup (public/private subnets with NAT)

(6B) - AWS VPC Peering

(7) - Web server

(8) - Database

(9) - Linux System / Application Monitoring, Performance Tuning, Profiling Methods & Tools

(10) - Trouble Shooting: Load, Throughput, Response time and Leaks

(11) - SSH key pairs, SSL Certificate, and SSL Handshake

(12) - Why is the database slow?

(13) - Is my web site down?

(14) - Is my server down?

(15) - Why is the server sluggish?

(16A) - Serving multiple domains using Virtual Hosts - Apache

(16B) - Serving multiple domains using server block - Nginx

(16C) - Reverse proxy servers and load balancers - Nginx

(17) - Linux startup process

(18) - phpMyAdmin with Nginx virtual host as a subdomain

(19) - How to SSH login without password?

(20) - Log Rotation

(21) - Monitoring Metrics

(22) - lsof

(23) - Wireshark introduction

(24) - User account management

(25) - Domain Name System (DNS)

(26) - NGINX SSL/TLS, Caching, and Session

(27) - Troubleshooting 5xx server errors

(28) - Linux Systemd: journalctl

(29) - Linux Systemd: FirewallD

(30) - Linux: SELinux

(31) - Linux: Samba

(0) - Linux Sys Admin's Day to Day tasks





Jenkins



Install

Configuration - Manage Jenkins - security setup

Adding job and build

Scheduling jobs

Managing_plugins

Git/GitHub plugins, SSH keys configuration, and Fork/Clone

JDK & Maven setup

Build configuration for GitHub Java application with Maven

Build Action for GitHub Java application with Maven - Console Output, Updating Maven

Commit to changes to GitHub & new test results - Build Failure

Commit to changes to GitHub & new test results - Successful Build

Adding code coverage and metrics

Jenkins on EC2 - creating an EC2 account, ssh to EC2, and install Apache server

Jenkins on EC2 - setting up Jenkins account, plugins, and Configure System (JAVA_HOME, MAVEN_HOME, notification email)

Jenkins on EC2 - Creating a Maven project

Jenkins on EC2 - Configuring GitHub Hook and Notification service to Jenkins server for any changes to the repository

Jenkins on EC2 - Line Coverage with JaCoCo plugin

Setting up Master and Slave nodes

Jenkins Build Pipeline & Dependency Graph Plugins

Jenkins Build Flow Plugin

Pipeline Jenkinsfile with Classic / Blue Ocean

Jenkins Setting up Slave nodes on AWS

Jenkins Q & A





Puppet



Puppet with Amazon AWS I - Puppet accounts

Puppet with Amazon AWS II (ssh & puppetmaster/puppet install)

Puppet with Amazon AWS III - Puppet running Hello World

Puppet Code Basics - Terminology

Puppet with Amazon AWS on CentOS 7 (I) - Master setup on EC2

Puppet with Amazon AWS on CentOS 7 (II) - Configuring a Puppet Master Server with Passenger and Apache

Puppet master /agent ubuntu 14.04 install on EC2 nodes

Puppet master post install tasks - master's names and certificates setup,

Puppet agent post install tasks - configure agent, hostnames, and sign request

EC2 Puppet master/agent basic tasks - main manifest with a file resource/module and immediate execution on an agent node

Setting up puppet master and agent with simple scripts on EC2 / remote install from desktop

EC2 Puppet - Install lamp with a manifest ('puppet apply')

EC2 Puppet - Install lamp with a module

Puppet variable scope

Puppet packages, services, and files

Puppet packages, services, and files II with nginx Puppet templates

Puppet creating and managing user accounts with SSH access

Puppet Locking user accounts & deploying sudoers file

Puppet exec resource

Puppet classes and modules

Puppet Forge modules

Puppet Express

Puppet Express 2

Puppet 4 : Changes

Puppet --configprint

Puppet with Docker

Puppet 6.0.2 install on Ubuntu 18.04





Chef



What is Chef?

Chef install on Ubuntu 14.04 - Local Workstation via omnibus installer

Setting up Hosted Chef server

VirtualBox via Vagrant with Chef client provision

Creating and using cookbooks on a VirtualBox node

Chef server install on Ubuntu 14.04

Chef workstation setup on EC2 Ubuntu 14.04

Chef Client Node - Knife Bootstrapping a node on EC2 ubuntu 14.04





Elasticsearch search engine, Logstash, and Kibana



Elasticsearch, search engine

Logstash with Elasticsearch

Logstash, Elasticsearch, and Kibana 4

Elasticsearch with Redis broker and Logstash Shipper and Indexer

Samples of ELK architecture

Elasticsearch indexing performance



Vagrant



VirtualBox & Vagrant install on Ubuntu 14.04

Creating a VirtualBox using Vagrant

Provisioning

Networking - Port Forwarding

Vagrant Share

Vagrant Rebuild & Teardown

Vagrant & Ansible





Big Data & Hadoop Tutorials



Hadoop 2.6 - Installing on Ubuntu 14.04 (Single-Node Cluster)

Hadoop 2.6.5 - Installing on Ubuntu 16.04 (Single-Node Cluster)

Hadoop - Running MapReduce Job

Hadoop - Ecosystem

CDH5.3 Install on four EC2 instances (1 Name node and 3 Datanodes) using Cloudera Manager 5

CDH5 APIs

QuickStart VMs for CDH 5.3

QuickStart VMs for CDH 5.3 II - Testing with wordcount

QuickStart VMs for CDH 5.3 II - Hive DB query

Scheduled start and stop CDH services

CDH 5.8 Install with QuickStarts Docker

Zookeeper & Kafka Install

Zookeeper & Kafka - single node single broker

Zookeeper & Kafka - Single node and multiple brokers

OLTP vs OLAP

Apache Hadoop Tutorial I with CDH - Overview

Apache Hadoop Tutorial II with CDH - MapReduce Word Count

Apache Hadoop Tutorial III with CDH - MapReduce Word Count 2

Apache Hadoop (CDH 5) Hive Introduction

CDH5 - Hive Upgrade to 1.3 to from 1.2

Apache Hive 2.1.0 install on Ubuntu 16.04

Apache HBase in Pseudo-Distributed mode

Creating HBase table with HBase shell and HUE

Apache Hadoop : Hue 3.11 install on Ubuntu 16.04

Creating HBase table with Java API

HBase - Map, Persistent, Sparse, Sorted, Distributed and Multidimensional

Flume with CDH5: a single-node Flume deployment (telnet example)

Apache Hadoop (CDH 5) Flume with VirtualBox : syslog example via NettyAvroRpcClient

List of Apache Hadoop hdfs commands

Apache Hadoop : Creating Wordcount Java Project with Eclipse Part 1

Apache Hadoop : Creating Wordcount Java Project with Eclipse Part 2

Apache Hadoop : Creating Card Java Project with Eclipse using Cloudera VM UnoExample for CDH5 - local run

Apache Hadoop : Creating Wordcount Maven Project with Eclipse

Wordcount MapReduce with Oozie workflow with Hue browser - CDH 5.3 Hadoop cluster using VirtualBox and QuickStart VM

Spark 1.2 using VirtualBox and QuickStart VM - wordcount

Spark Programming Model : Resilient Distributed Dataset (RDD) with CDH

Apache Spark 2.0.2 with PySpark (Spark Python API) Shell

Apache Spark 2.0.2 tutorial with PySpark : RDD

Apache Spark 2.0.0 tutorial with PySpark : Analyzing Neuroimaging Data with Thunder

Apache Spark Streaming with Kafka and Cassandra

Apache Spark 1.2 with PySpark (Spark Python API) Wordcount using CDH5

Apache Spark 1.2 Streaming

Apache Drill with ZooKeeper install on Ubuntu 16.04 - Embedded & Distributed

Apache Drill - Query File System, JSON, and Parquet

Apache Drill - HBase query

Apache Drill - Hive query

Apache Drill - MongoDB query





Redis In-Memory Database



Redis vs Memcached

Redis 3.0.1 Install

Setting up multiple server instances on a Linux host

Redis with Python

ELK : Elasticsearch with Redis broker and Logstash Shipper and Indexer



GCP (Google Cloud Platform)



GCP: Creating an Instance

GCP: gcloud compute command-line tool

GCP: Deploying Containers

GCP: Kubernetes Quickstart

GCP: Deploying a containerized web application via Kubernetes

GCP: Django Deploy via Kubernetes I (local)

GCP: Django Deploy via Kubernetes II (GKE)





AWS (Amazon Web Services)



AWS : EKS (Elastic Container Service for Kubernetes)

AWS : Creating a snapshot (cloning an image)

AWS : Attaching Amazon EBS volume to an instance

AWS : Adding swap space to an attached volume via mkswap and swapon

AWS : Creating an EC2 instance and attaching Amazon EBS volume to the instance using Python boto module with User data

AWS : Creating an instance to a new region by copying an AMI

AWS : S3 (Simple Storage Service) 1

AWS : S3 (Simple Storage Service) 2 - Creating and Deleting a Bucket

AWS : S3 (Simple Storage Service) 3 - Bucket Versioning

AWS : S3 (Simple Storage Service) 4 - Uploading a large file

AWS : S3 (Simple Storage Service) 5 - Uploading folders/files recursively

AWS : S3 (Simple Storage Service) 6 - Bucket Policy for File/Folder View/Download

AWS : S3 (Simple Storage Service) 7 - How to Copy or Move Objects from one region to another

AWS : S3 (Simple Storage Service) 8 - Archiving S3 Data to Glacier

AWS : Creating a CloudFront distribution with an Amazon S3 origin

AWS : Creating VPC with CloudFormation

WAF (Web Application Firewall) with preconfigured CloudFormation template and Web ACL for CloudFront distribution

AWS : CloudWatch & Logs with Lambda Function / S3

AWS : Lambda Serverless Computing with EC2, CloudWatch Alarm, SNS

AWS : Lambda and SNS - cross account

AWS : CLI (Command Line Interface)

AWS : CLI (ECS with ALB & autoscaling)

AWS : ECS with cloudformation and json task definition

AWS : AWS Application Load Balancer (ALB) and ECS with Flask app

AWS : Load Balancing with HAProxy (High Availability Proxy)

AWS : VirtualBox on EC2

AWS : NTP setup on EC2

AWS: jq with AWS

AWS : AWS & OpenSSL : Creating / Installing a Server SSL Certificate

AWS : OpenVPN Access Server 2 Install

AWS : VPC (Virtual Private Cloud) 1 - netmask, subnets, default gateway, and CIDR

AWS : VPC (Virtual Private Cloud) 2 - VPC Wizard

AWS : VPC (Virtual Private Cloud) 3 - VPC Wizard with NAT

AWS : DevOps / Sys Admin Q & A (VI) - AWS VPC setup (public/private subnets with NAT)

AWS : OpenVPN Protocols : PPTP, L2TP/IPsec, and OpenVPN

AWS : Autoscaling group (ASG)

AWS : Setting up Autoscaling Alarms and Notifications via CLI and Cloudformation

AWS : Adding a SSH User Account on Linux Instance

AWS : Windows Servers - Remote Desktop Connections using RDP

AWS : Scheduled stopping and starting an instance - python & cron

AWS : Detecting stopped instance and sending an alert email using Mandrill smtp

AWS : Elastic Beanstalk with NodeJS

AWS : Elastic Beanstalk Inplace/Rolling Blue/Green Deploy

AWS : Identity and Access Management (IAM) Roles for Amazon EC2

AWS : Identity and Access Management (IAM) Policies, sts AssumeRole, and delegate access across AWS accounts

AWS : Identity and Access Management (IAM) sts assume role via aws cli2

AWS : Creating IAM Roles and associating them with EC2 Instances in CloudFormation

AWS Identity and Access Management (IAM) Roles, SSO(Single Sign On), SAML(Security Assertion Markup Language), IdP(identity provider), STS(Security Token Service), and ADFS(Active Directory Federation Services)

AWS : Amazon Route 53

AWS : Amazon Route 53 - DNS (Domain Name Server) setup

AWS : Amazon Route 53 - subdomain setup and virtual host on Nginx

AWS Amazon Route 53 : Private Hosted Zone

AWS : SNS (Simple Notification Service) example with ELB and CloudWatch

AWS : Lambda with AWS CloudTrail

AWS : SQS (Simple Queue Service) with NodeJS and AWS SDK

AWS : Redshift data warehouse

AWS : CloudFormation - templates, change sets, and CLI

AWS : CloudFormation Bootstrap UserData/Metadata

AWS : CloudFormation - Creating an ASG with rolling update

AWS : Cloudformation Cross-stack reference

AWS : OpsWorks

AWS : Network Load Balancer (NLB) with Autoscaling group (ASG)

AWS CodeDeploy : Deploy an Application from GitHub

AWS EC2 Container Service (ECS)

AWS EC2 Container Service (ECS) II

AWS Hello World Lambda Function

AWS Lambda Function Q & A

AWS Node.js Lambda Function & API Gateway

AWS API Gateway endpoint invoking Lambda function

AWS API Gateway invoking Lambda function with Terraform

AWS API Gateway invoking Lambda function with Terraform - Lambda Container

Amazon Kinesis Streams

Kinesis Data Firehose with Lambda and ElasticSearch

Amazon DynamoDB

Amazon DynamoDB with Lambda and CloudWatch

Loading DynamoDB stream to AWS Elasticsearch service with Lambda

Amazon ML (Machine Learning)

Simple Systems Manager (SSM)

AWS : RDS Connecting to a DB Instance Running the SQL Server Database Engine

AWS : RDS Importing and Exporting SQL Server Data

AWS : RDS PostgreSQL & pgAdmin III

AWS : RDS PostgreSQL 2 - Creating/Deleting a Table

AWS : MySQL Replication : Master-slave

AWS : MySQL backup & restore

AWS RDS : Cross-Region Read Replicas for MySQL and Snapshots for PostgreSQL

AWS : Restoring Postgres on EC2 instance from S3 backup

AWS : Q & A

AWS : Security

AWS : Security groups vs. network ACLs

AWS : Scaling-Up

AWS : Networking

AWS : Single Sign-on (SSO) with Okta

AWS : JIT (Just-in-Time) with Okta





Powershell 4 Tutorial



Powersehll : Introduction

Powersehll : Help System

Powersehll : Running commands

Powersehll : Providers

Powersehll : Pipeline

Powersehll : Objects

Powershell : Remote Control

Windows Management Instrumentation (WMI)

How to Enable Multiple RDP Sessions in Windows 2012 Server

How to install and configure FTP server on IIS 8 in Windows 2012 Server

How to Run Exe as a Service on Windows 2012 Server

SQL Inner, Left, Right, and Outer Joins





Git/GitHub Tutorial



One page express tutorial for GIT and GitHub

Installation

add/status/log

commit and diff

git commit --amend

Deleting and Renaming files

Undoing Things : File Checkout & Unstaging

Reverting commit

Soft Reset - (git reset --soft <SHA key>)

Mixed Reset - Default

Hard Reset - (git reset --hard <SHA key>)

Creating & switching Branches

Fast-forward merge

Rebase & Three-way merge

Merge conflicts with a simple example

GitHub Account and SSH

Uploading to GitHub

GUI

Branching & Merging

Merging conflicts

GIT on Ubuntu and OS X - Focused on Branching

Setting up a remote repository / pushing local project and cloning the remote repo

Fork vs Clone, Origin vs Upstream

Git/GitHub Terminologies

Git/GitHub via SourceTree II : Branching & Merging

Git/GitHub via SourceTree III : Git Work Flow

Git/GitHub via SourceTree IV : Git Reset

Git wiki - quick command reference






Subversion

Subversion Install On Ubuntu 14.04

Subversion creating and accessing I

Subversion creating and accessing II








Contact

BogoToBogo
contactus@bogotobogo.com

Follow Bogotobogo

About Us

contactus@bogotobogo.com

YouTubeMy YouTube channel
Pacific Ave, San Francisco, CA 94115

Pacific Ave, San Francisco, CA 94115

Copyright © 2024, bogotobogo
Design: Web Master