PROJECT: Automating Java CI/CD pipeline with Jenkins, Git, GitHub, Maven, Check style, SonarQube, Nexus, AWS EC2 and Slack Notifications":

Scenario:

🚀 The scenario involves product development with an Agile SDLC in motion.

🚀 A team of developers will regularly make code changes, resulting in multiple code changes every day.

🚀 This code needs to be tested, as it is what builds the product.

🚀 In an enterprise, a separate builder release team is responsible for building, testing, and releasing the code. In small industries, developers may be responsible for merging and integrating the code.

🚀 Regular code changes, also called commits or pull requests, occur.

🚀 Developers typically depend on the build and release team to test the code and move it to the next level in the release cycle, which may result in delayed detection of bugs or errors.

🚀 Accumulation of bugs and errors may lead to a time-consuming rework process and missed deadlines.

🚀 To solve this problem, there should be regular build and test for every commit.

🚀 An automated build and release process should be in place to make this possible.

🚀 Whenever there is a build and test of the code, the developer should get notified automatically if there are any failures or errors.

🚀 This will allow for prompt bug fixing and prevent delays in the development process.

Tools:

🛠️ For this project, we will be using several tools, starting with Jenkins. Jenkins will serve as our continuous integration server and main hero.

🛠️ Our Jenkins hero will also need an assistant tool - a version control system. We'll be using Git and GitHub as the remote repository, and we'll have Java code in it. To build that Java code, we'll need the build tool Maven.

🛠️ We'll also be using Checkstyle for code analysis. It's a simple code analysis tool, but we have more sidekicks to assist us.

🛠️ Slack will be our notification tool, and we can integrate e-mail notifications as well. We'll use Nexus Sonatype to store our artifact and download dependencies for Maven. It will be our software repository.

🛠️ We'll be going deeper into code analysis using Sonarqube server. We'll scan our code with the Sonarqube scanner and Checkstyle and then publish the results into the Sonarqube server dashboard.

🛠️ To set up our Jenkins server, Nexus server, and Sonarqube server, we'll be using AWS EC2 instances. We'll set up these servers on the EC2 instances.

Architecture:

The flow of Executions:

🔆 Log into the EWS account

🔆 Create key pair for EC2 instance

🔆 Create a security group for Jenkins, Nexus, and Sonarqube server

🔆 Launch EC2 instance with user data script

🔆 Conduct post-installation steps:

🔆 Set up Jenkins user

🔆 Install necessary plugins

🔆 Set up Nexus login and create repositories

🔆 Test installation

🔆 Create a GitHub repository and migrate code

🔆 Integrate the GitHub repository with VS Code and test

🔆 Write code for the first job (build job) to integrate with Nexus and fetch dependencies

🔆 Create GitHub webhook for automatic job triggering

🔆 Integrate with the Sonarqube server and upload test results

🔆 Upload artefact to Nexus server

🔆 Create a Slack notification system for pipeline changes

Step-1 Setup all necessary tools from the Repo:

Step-2 Launch EC2 Server with all configuration:

Launch Three Server:

Jenkins Server
Nexus Server
Sonar Server

Now Try to Access Jenkins Server:

Now Try to Access Nexus Server:

Now Try to access Sonar Server:

Step-3 Log in to the Jenkins server and install plugins:

Here We will install Maven Integration, GitHub Integration, Nexus Artifact Uploader, SonarCubeScanner, Slack Notification and Build Timestamp.

Step-4 Setup Nexus Server:

Create a Repository here:

Maven-2 Hosted and Maven-2 Proxy

So here we create total of four repo and in Vpro Group we add all three repo to create the Group:

Step-5 Git Code Migration:

Step-1 Create a new Repository in your github account:

Make it Private:

here we have cloned the branch name ci-jenkins :

Now cat the config file.

Now replace the URL with our Repo URL :

Now you can see the URL Got changed.

Now create a New branch and enter it.

Now push my main branch and ci-jenkins to my github:

Now Verify it with Github:

Step-6 Build Job with Nexus Repo:

Set the Global Tool Configuration like JDK and Maven:

Also, Save the Nexus Login Credentials:

Go to Manage Credentials, Add Credentials with Username and Password of Nexus:

also add credentials for Github account

Now Create a Pipeline Code for creating a JOB:

pipeline {
    agent any

    environment {
        SNAP_REPO = 'vprofile-snapshot'
        NEXUS_USER = 'admin'
        NEXUS_PASS = '#############'
        RELEASE_REPO = 'vprofile-release'
        CENTRAL_REPO = 'vpro-maven-central'
        NEXUSIP = '172.31.45.30'
        NEXUSPORT = '8081'
        NEXUS_GRP_REPO = 'vpro-maven-group'
        NEXUS_LOGIN = 'nexuslogin'
    }

    tools {
        maven 'MAVEN3'
        jdk 'oraclejdk-8'
    }

    stages {
        stage('clone repository') {
            steps {
                dir('/var/lib/jenkins/workspace/vprofile-ci-pipeline') {
                    git branch: 'ci-jenkins',
                        url: 'https://github.com/abhijeetpoonia/vprociproject.git',
                        credentialsId: 'd950b127-b556-4753-8b8d-0e31cc5f1d41'
                }
            }
        }

        stage('Build') {
            steps {
                sh 'mvn -s settings.xml -DskipTests -Dusername=${NEXUS_USER} -Dpassword=${NEXUS_PASS} install'
            }

            post {
                success {
                    echo "Now Archiving."
                    archiveArtifacts artifacts: '**/*.war'
                }
            }
        }

        stage('Test'){
            steps {
                sh 'mvn -s settings.xml test'
            }

        }

        stage('Checkstyle Analysis'){
            steps {
                sh 'mvn -s settings.xml checkstyle:checkstyle'
            }
        }
    }
}

Now build it .

Step-7 Code Analysis with Sonarcube:

Step-1 Go to Workspace ------> target-------> Here you will find checkstyle-result.xml and surefire-reports but it is not in readable format:

Now Login to your Jenkins and write code so that it can scan sode and give analysis:

Got Global Tool Configuration and find Sonarcube:

2 Go to Configuration System and enter the detail of the Sonarcube server:

here I have used the public ip of my sonarcube server:

Now Add a token for this first Generate the Token , Go to Sonarcube Server:

Got to account----> security------> Generate

Now Write a code so that it can generate a report on Sonarcube:

 pipeline {
    agent any

    environment {
        SNAP_REPO = 'vprofile-snapshot'
        NEXUS_USER = 'admin'
        NEXUS_PASS = '################'
        RELEASE_REPO = 'vprofile-release'
        CENTRAL_REPO = 'vpro-maven-central'
        NEXUSIP = '172.31.45.30'
        NEXUSPORT = '8081'
        NEXUS_GRP_REPO = 'vpro-maven-group'
        NEXUS_LOGIN = 'nexuslogin'
        SONARSERVER = 'sonarserver'
        SONARSCANNER = 'sonarscanner'
    }

    tools {
        maven 'MAVEN3'
        jdk 'oraclejdk-11'
    }

    stages {
        stage('clone repository') {
            steps {
                dir('/var/lib/jenkins/workspace/vprofile-ci-pipeline') {
                    git branch: 'ci-jenkins',
                        url: 'https://github.com/abhijeetpoonia/vprociproject.git',
                        credentialsId: 'd950b127-b556-4753-8b8d-0e31cc5f1d41'
                }
            }
        }

        stage('Build') {
            steps {
                sh 'mvn -s settings.xml -DskipTests -Dusername=${NEXUS_USER} -Dpassword=${NEXUS_PASS} install'
            }

            post {
                success {
                    echo "Now Archiving."
                    archiveArtifacts artifacts: '**/*.war'
                }
            }
        }

        stage('Test'){
            steps {
                sh 'mvn -s settings.xml test'
            }

        }

        stage('Checkstyle Analysis'){
            steps {
                sh 'mvn -s settings.xml checkstyle:checkstyle'
            }
        }

        stage('Sonar Analysis') {
            environment {
                scannerHome = tool "${SONARSCANNER}"
            }
            steps {
                withSonarQubeEnv("${SONARSERVER}") {
                    sh '''${scannerHome}/bin/sonar-scanner -Dsonar.projectKey=vprofile \
                    -Dsonar.projectName=vprofile \
                    -Dsonar.projectVersion=1.0 \
                    -Dsonar.sources=src/ \
                    -Dsonar.java.binaries=target/test-classes/com/visualpathit/account/controllerTest/ \
                    -Dsonar.junit.reportsPath=target/surefire-reports/ \
                    -Dsonar.jacoco.reportsPath=target/jacoco.exec \
                    -Dsonar.java.checkstyle.reportPaths=target/checkstyle-result.xml'''
                }
            }
        }
    }
}

Note: Here I was using Java 1.8 which may not be compatible with your Sonarcube verson so just update it to version-11 and then update the global tool and update $PATH

Now Go to Sonarcube and Check your Project:

Step-8 Setup Sonar Quality Gates & publish artefact to Nexus Repo:

First go to the sonar server , Project setting and create a quality Gate:

Add Conditions because we find many bugs here:

now go to project -----> setting------>qualityGate----->and select one you have created:

Now we have to set up Webhook so that Sonarcube will get to know about Jenkins:

Got to setting-------> webhook and create:

Now Write a code :

 pipeline {
    agent any

    environment {
        SNAP_REPO = 'vprofile-snapshot'
        NEXUS_USER = 'admin'
        NEXUS_PASS = '###############'
        RELEASE_REPO = 'vprofile-release'
        CENTRAL_REPO = 'vpro-maven-central'
        NEXUSIP = '172.31.45.30'
        NEXUSPORT = '8081'
        NEXUS_GRP_REPO = 'vpro-maven-group'
        NEXUS_LOGIN = 'nexuslogin'
        SONARSERVER = 'sonarserver'
        SONARSCANNER = 'sonarscanner'
    }

    tools {
        maven 'MAVEN3'
        jdk 'oraclejdk-11'
    }

    stages {
        stage('clone repository') {
            steps {
                dir('/var/lib/jenkins/workspace/vprofile-ci-pipeline') {
                    git branch: 'ci-jenkins',
                        url: 'https://github.com/abhijeetpoonia/vprociproject.git',
                        credentialsId: 'd950b127-b556-4753-8b8d-0e31cc5f1d41'
                }
            }
        }

        stage('Build') {
            steps {
                sh 'mvn -s settings.xml -DskipTests -Dusername=${NEXUS_USER} -Dpassword=${NEXUS_PASS} install'
            }

            post {
                success {
                    echo "Now Archiving."
                    archiveArtifacts artifacts: '**/*.war'
                }
            }
        }

        stage('Test'){
            steps {
                sh 'mvn -s settings.xml test'
            }

        }

        stage('Checkstyle Analysis'){
            steps {
                sh 'mvn -s settings.xml checkstyle:checkstyle'
            }
        }

        stage('Sonar Analysis') {
            environment {
                scannerHome = tool "${SONARSCANNER}"
            }
            steps {
                withSonarQubeEnv("${SONARSERVER}") {
                    sh '''${scannerHome}/bin/sonar-scanner -Dsonar.projectKey=vprofile \
                    -Dsonar.projectName=vprofile \
                    -Dsonar.projectVersion=1.0 \
                    -Dsonar.sources=src/ \
                    -Dsonar.java.binaries=target/test-classes/com/visualpathit/account/controllerTest/ \
                    -Dsonar.junit.reportsPath=target/surefire-reports/ \
                    -Dsonar.jacoco.reportsPath=target/jacoco.exec \
                    -Dsonar.java.checkstyle.reportPaths=target/checkstyle-result.xml'''
                }
            }
        }

        stage('Quality Gate') {
            steps {
                timeout(time: 1, unit: 'HOURS') {
                    // Parameter indicates whether to set pipeline to UNSTABLE if Quality Gate fails
                    // true = set pipeline to UNSTABLE, false = don't
                    waitForQualityGate abortPipeline: true
                }
            }
        }
    }
}

and Run it :

Now Publish artefact to Nexus Repo:

Now firstly look at the artifact:

here you can see the .war(bottom) file that needs to send to Nexus:

Go to manage jenkins----->configure System-------> build timestamp

Now write Code:

  pipeline {
    agent any

    environment {
        SNAP_REPO = 'vprofile-snapshot'
        NEXUS_USER = 'admin'
        NEXUS_PASS = '################'
        RELEASE_REPO = 'vprofile-release'
        CENTRAL_REPO = 'vpro-maven-central'
        NEXUSIP = '172.31.45.30'
        NEXUSPORT = '8081'
        NEXUS_GRP_REPO = 'vpro-maven-group'
        NEXUS_LOGIN = 'nexuslogin'
        SONARSERVER = 'sonarserver'
        SONARSCANNER = 'sonarscanner'
    }

    tools {
        maven 'MAVEN3'
        jdk 'oraclejdk-11'
    }

    stages {
        stage('clone repository') {
            steps {
                dir('/var/lib/jenkins/workspace/vprofile-ci-pipeline') {
                    git branch: 'ci-jenkins',
                        url: 'https://github.com/abhijeetpoonia/vprociproject.git',
                        credentialsId: 'd950b127-b556-4753-8b8d-0e31cc5f1d41'
                }
            }
        }

        stage('Build') {
            steps {
                sh 'mvn -s settings.xml -DskipTests -Dusername=${NEXUS_USER} -Dpassword=${NEXUS_PASS} install'
            }

            post {
                success {
                    echo "Now Archiving."
                    archiveArtifacts artifacts: '**/*.war'
                }
            }
        }

        stage('Test'){
            steps {
                sh 'mvn -s settings.xml test'
            }

        }

        stage('Checkstyle Analysis'){
            steps {
                sh 'mvn -s settings.xml checkstyle:checkstyle'
            }
        }

        stage('Sonar Analysis') {
            environment {
                scannerHome = tool "${SONARSCANNER}"
            }
            steps {
                withSonarQubeEnv("${SONARSERVER}") {
                    sh '''${scannerHome}/bin/sonar-scanner -Dsonar.projectKey=vprofile \
                    -Dsonar.projectName=vprofile \
                    -Dsonar.projectVersion=1.0 \
                    -Dsonar.sources=src/ \
                    -Dsonar.java.binaries=target/test-classes/com/visualpathit/account/controllerTest/ \
                    -Dsonar.junit.reportsPath=target/surefire-reports/ \
                    -Dsonar.jacoco.reportsPath=target/jacoco.exec \
                    -Dsonar.java.checkstyle.reportPaths=target/checkstyle-result.xml'''
                }
            }
        }
      stage("UploadArtifact"){
            steps{
                nexusArtifactUploader(
                  nexusVersion: 'nexus3',
                  protocol: 'http',
                  nexusUrl: "${NEXUSIP}:${NEXUSPORT}",
                  groupId: 'QA',
                  version: "${env.BUILD_ID}-${env.BUILD_TIMESTAMP}",
                  repository: "${RELEASE_REPO}",
                  credentialsId: "${NEXUS_LOGIN}",
                  artifacts: [
                    [artifactId: 'vproapp',
                     classifier: '',
                     file: 'target/vprofile-v2.war',
                     type: 'war']
                  ]
                )
            }
        }
    }
}

You can download it with Given Path on right side:

Step-9 Slack Notification:

Create a new workspace in slack:

In order to authenticate workspace by jenkins we need to generate token:

Search Slack apps on browser:

search jenkins:

Now add it to Slack:

Just copy the token from Step-3 and save the setting at bottom of this page.

Now go to manage jenkins------->configure system and find slack there add workspace name and credential as token.

and test the connection:

Verify it on Slack:

Now write code :

 def COLOR_MAP = [
    'SUCCESS': 'good', 
    'FAILURE': 'danger',
]

pipeline {
    agent any

    environment {
        SNAP_REPO = 'vprofile-snapshot'
        NEXUS_USER = 'admin'
        NEXUS_PASS = '###############'
        RELEASE_REPO = 'vprofile-release'
        CENTRAL_REPO = 'vpro-maven-central'
        NEXUSIP = '172.31.45.30'
        NEXUSPORT = '8081'
        NEXUS_GRP_REPO = 'vpro-maven-group'
        NEXUS_LOGIN = 'nexuslogin'
        SONARSERVER = 'sonarserver'
        SONARSCANNER = 'sonarscanner'
    }

    tools {
        maven 'MAVEN3'
        jdk 'oraclejdk-11'
    }

    stages {
        stage('clone repository') {
            steps {
                dir('/var/lib/jenkins/workspace/vprofile-ci-pipeline') {
                    git branch: 'ci-jenkins',
                        url: 'https://github.com/abhijeetpoonia/vprociproject.git',
                        credentialsId: 'd950b127-b556-4753-8b8d-0e31cc5f1d41'
                }
            }
        }

        stage('Build') {
            steps {
                sh 'mvn -s settings.xml -DskipTests -Dusername=${NEXUS_USER} -Dpassword=${NEXUS_PASS} install'
            }

            post {
                success {
                    echo "Now Archiving."
                    archiveArtifacts artifacts: '**/*.war'
                }
            }
        }

        stage('Test'){
            steps {
                sh 'mvn -s settings.xml test'
            }

        }

        stage('Checkstyle Analysis'){
            steps {
                sh 'mvn -s settings.xml checkstyle:checkstyle'
            }
        }

        stage('Sonar Analysis') {
            environment {
                scannerHome = tool "${SONARSCANNER}"
            }
            steps {
                withSonarQubeEnv("${SONARSERVER}") {
                    sh '''${scannerHome}/bin/sonar-scanner -Dsonar.projectKey=vprofile \
                    -Dsonar.projectName=vprofile \
                    -Dsonar.projectVersion=1.0 \
                    -Dsonar.sources=src/ \
                    -Dsonar.java.binaries=target/test-classes/com/visualpathit/account/controllerTest/ \
                    -Dsonar.junit.reportsPath=target/surefire-reports/ \
                    -Dsonar.jacoco.reportsPath=target/jacoco.exec \
                    -Dsonar.java.checkstyle.reportPaths=target/checkstyle-result.xml'''
                }
            }
        }

        stage("UploadArtifact"){
            steps{
                nexusArtifactUploader(
                  nexusVersion: 'nexus3',
                  protocol: 'http',
                  nexusUrl: "${NEXUSIP}:${NEXUSPORT}",
                  groupId: 'QA',
                  version: "${env.BUILD_ID}-${env.BUILD_TIMESTAMP}",
                  repository: "${RELEASE_REPO}",
                  credentialsId: "${NEXUS_LOGIN}",
                  artifacts: [
                    [artifactId: 'vproapp',
                     classifier: '',
                     file: 'target/vprofile-v2.war',
                     type: 'war']
                  ]
                )
            }
        }
    }
    post {
        always {
            echo 'Slack Notifications.'
            slackSend channel: '#jenkinscicd',
                color: COLOR_MAP[currentBuild.currentResult],
                message: "*${currentBuild.currentResult}:* Job ${env.JOB_NAME} build ${env.BUILD_NUMBER} \n More info at: ${env.BUILD_URL}"
        }
    }
}

Now Check the notification on Slack:

Click on this URL and access it.

Abhi blog