Monday, June 24, 2024

Dockerfile for MageAI

 Dockerfile for MageAI


MageAI Document Link - https://docs.mage.ai/getting-started/setup#docker-compose-template


    You can use above link to run MageAI with docker-compose and if you end up on error like "Illegal Instruction" then you will have to use custom dockerfile.

1] Create Dockerfile

# mkdir mageai; cd mageai

# vi Dockerfile
FROM mageai/mageai:latest
RUN pip uninstall -y polars && \
    pip install "polars-lts-cpu>=0.19.0,<0.19.2"
EXPOSE 6789
CMD ["/app/run_app.sh"]

2] Save above dockerfile and build docker image

# docker build -t localmageai .

3] Run docker container on port 6789

# docker run -d -p 6789:6789 --name localmageai locamageai:latest

4] Open browser and try to access http://localhost:6789.  You should be able to run MageAI UI successfully.

Thursday, December 28, 2023

Enable SSH in Azure Webapp Container

 Enable SSH in Azure Webapp Container


    This article will help you enable ssh in Azure Webapp container.

- Create a file called "init.sh" at the same location where your Dockerfile is and add below content to it.

#!/bin/bash
set -e

echo "Starting SSH ..."
service ssh start

exec "$@"

- Create a file called "sshd_config", this file will have basic SSH configuration and port for SSH
and add below content.

Port 2222 ListenAddress 0.0.0.0 LoginGraceTime 180 X11Forwarding yes Ciphers aes128-cbc,3des-cbc,aes256-cbc,aes128-ctr,aes192-ctr,aes256-ctr MACs hmac-sha1,hmac-sha1-96 StrictModes yes SyslogFacility DAEMON PasswordAuthentication yes PermitEmptyPasswords no PermitRootLogin yes Subsystem sftp internal-sftp

- Install OpenSSH service, add below lines in Dockerfile.

# Install and Run SSH
ENV SSH_PASSWD "root:Docker!"
RUN apt-get update
RUN apt-get install -y dialog \
    && apt-get update \
    && apt-get install -y openssh-server \
    && echo "$SSH_PASSWD" | chpasswd

COPY ./sshd_config /etc/ssh/
COPY ./init.sh /usr/local/bin/
RUN chmod u+x /usr/local/bin/init.sh
ENTRYPOINT ["init.sh"]

- Build Docker image and deploy it on Azure webapp, to validate SSH navigate to Azure Portal.


Login into your Azure Portal–> Navigate to App services–> Enter into your deployed App service–>
Click on SSH(from the side pane)–> Click on Go


It will open another tab in browser and login to container.





Friday, July 21, 2023

Github Action workflow to deploy flask application on Google Kubernetes Engine (GKE)

 Deploy application on GKE using Github Actions


Follow the below steps to setup Github Actions workflow.

Prerequisite:

- Create service account in GCP and generate key file.

- Convert key file into base64 format and using the output to create secret variable in github repo.

Steps:

- In your github repo create .github/workflow directory

- Under workflow directory create a file called deploy.yml and paste below code into it.

name: Build-Deploy

on:

  push:

    branches: [ "anup_gke" ]

env:

  PROJECT_ID: <google-project-id> # ${{ secrets.GKE_PROJECT }}

  GKE_CLUSTER: <GKE-Cluster-name>    # TODO: update to cluster name

  GKE_ZONE: us-central1-a   # TODO: update to cluster zone

  DEPLOYMENT_NAME: gke-test # TODO: update to deployment name

  IMAGE_FLASK: <image name>

  IMAGE_NGINX: <image name>

jobs:

  build-deploy-gke:

    name: Login, Build, Publish, and Deploy

    runs-on: ubuntu-latest

    steps:

    - name: Checkout

      uses: actions/checkout@v3

    - id: 'auth'

      name: Google Authentication

      uses: 'google-github-actions/auth@v1'

      with:

        credentials_json: '${{ secrets.GKE_SA_KEY }}'

    # Setup gcloud CLI

    - uses: google-github-actions/setup-gcloud@v1

      name: Setup gcloud CLI

    # Configure Docker to use the gcloud command-line tool as a credential

    # helper for authentication

    - name: 'Configure Docker to use the gcloud command-line tool'

      run: |-

        gcloud --quiet auth configure-docker

    # Build the Docker image for mvp-flask

    - name: Build Docker Image of mvp-flask

      run: |-

        cd source/mvp-flask

        docker build -t gcr.io/$PROJECT_ID/$IMAGE_FLASK:$GITHUB_RUN_ID .

    # Build the Docker image for mvp-nginx

    - name: Build Docker Image of mvp-nginx

      run: |-

        cd source/nginx

        docker build -t gcr.io/$PROJECT_ID/$IMAGE_NGINX:$GITHUB_RUN_ID .

    - name: Push docker images to GCR

      run: |-

        docker push gcr.io/$PROJECT_ID/$IMAGE_FLASK:$GITHUB_RUN_ID

        docker push gcr.io/$PROJECT_ID/$IMAGE_NGINX:$GITHUB_RUN_ID

    - name: 'Get GKE Credentails'

      id: 'get-credentials'

      uses: 'google-github-actions/get-gke-credentials@v1'

      with:

        cluster_name: ${{ env.GKE_CLUSTER }}

        location: ${{ env.GKE_ZONE }}

    - name: Deploy Docker Image on GKE Cluster

      run: |-

        kubectl set image deployment.apps/mvp-flask flask=gcr.io/<google-project-id>/$IMAGE_FLASK:$GITHUB_RUN_ID

        kubectl set image deployment.apps/mvp-nginx nginx=gcr.io/<google-project-id>/$IMAGE_NGINX:$GITHUB_RUN_ID

    - name: command

      run: |-

        kubectl get all

        echo "###"

        kubectl describe deployment.apps/mvp-flask

        echo "###"

        kubectl describe deployment.apps/mvp-nginx

Tuesday, August 30, 2022

Nginx location Regex Expression

 Nginx Regex Expression


Nginx location block allow you to route request to particular location in file system or particular url.

Below example is to show how to route domain name and number to domain.com:port.


server {

  listen 80;

  server_name anup.co.in;

  root /usr/share/nginx/html;

   location ~ "/app/lck/([a-z0-9\-\.]+)/([0-9]+)" {

      return 301 http://$1:$2;

   }

}


    Above highlighted part will redirect as follows -

http://anup.co.in/app/lck/google.com/8080  --> http://google.com:8080

Friday, March 11, 2022

How to run Docker inside Docker using Dockerfile

 How to run Docker inside Docker using Dockerfile


In some cases we want to run docker command inside docker container, we can do that by mapping docker.sock volume while running container.  The other option is to use your Dockerfile.

1] Here is Dockerfile -

FROM ubuntu:18.04

#Install Docker

RUN apt-get update

RUN apt-get -y install apt-transport-https

RUN apt-get -y install ca-certificates

RUN apt-get -y install curl

RUN apt-get -y install gnupg2

RUN apt-get -y install software-properties-common

ENV APT_KEY_DONT_WARN_ON_DANGEROUS_USAGE=DontWarn

RUN curl -fsSL https://download.docker.com/linux/ubuntu/gpg | (OUT=$(apt-key add - 2>&1) || echo $OUT)

RUN add-apt-repository --yes "deb [arch=amd64] https://download.docker.com/linux/ubuntu bionic stable"

RUN apt-get update

RUN echo "deb http://security.ubuntu.com/ubuntu xenial-security main" >> /etc/apt/sources.list; apt-get update

RUN apt-get -y install build-essential

RUN apt-get -y install docker-ce docker-ce-cli containerd.io

CMD ["tail", "-f", "/dev/null"]


2] Build docker image.

# docker build -t docker-in-docker:latest .



3] Run docker container from above docker image.
# docker run -d --name dockerINdocker docker-in-docker:latest

4] Enter into docker container and confirm docker version as per below image.



Tuesday, August 24, 2021

Download large file from Google Drive using wget on terminal

 Download large file from Google Drive using wget on terminal


       To download large file from Google Drive use following steps.


1] Share file publicly and Copy share URL.

Example share URL - 

https://drive.google.com/file/d/1tcthANUPNgyho7X-5HPDuUAiEfTfw5/view?usp=sharing


2] Extract Field ID from above share URL as below.

https://drive.google.com/file/d/1tcthANUPNgyho7X-5HPDuUAiEfTfw5/view?usp=sharing

Field ID is - 1tcthANUPNgyho7X-5HPDuUAiEfTfw5


3] Go to terminal and paste following command.

wget --load-cookies /tmp/cookies.txt "https://docs.google.com/uc?export=download&confirm=$(wget --quiet --save-cookies /tmp/cookies.txt --keep-session-cookies --no-check-certificate 'https://docs.google.com/uc?export=download&id=FIELDID' -O- | sed -rn 's/.*confirm=([0-9A-Za-z_]+).*/\1\n/p')&id=FIELDID" -O FILENAME && rm -rf /tmp/cookies.txt


Here, Replace FIELDID and FILENAME as per your file.


Let me know how it goes.

Wednesday, March 31, 2021

Nginx Cookbook

 Nginx Cookbook

1] Wildcard for Nginx location

I have multiple API running on server to access them through I have to add multiple location block as below.

My goal is to add single location block for all API's.

server { listen 80; server_name www.anup.co.in; location / { proxy_pass http://localhost:3000; } location /getHighscores { proxy_pass http://localhost:3000/getHighscores; } location /auth/google { proxy_pass http://localhost:3000/auth/google; } location /auth/google/redirect { proxy_pass http://localhost:3000/auth/google/redirect; } location /auth/login/success { proxy_pass http://localhost:3000/auth/login/success; } location /auth/login/failed { proxy_pass http://localhost:3000/auth/login/failed; } location /auth/logout { proxy_pass http://localhost:3000/auth/logout; } }

Solution:

server { listen 80; server_name www.anup.co.in; location / { proxy_pass http://localhost:3000; } location ~ ^/(.*)$ { proxy_pass http://localhost:3000/$1; } }