devops – Little Big Extra http://littlebigextra.com A technology blog covering topics on Java, Scala, Docker, AWS, BigData, DevOps and much more to come. Do it yourself instructions for complex to simple problems for a novice to an expert. Mon, 01 Apr 2019 14:18:04 +0000 en-US hourly 1 https://wordpress.org/?v=5.1.1 http://littlebigextra.com/wp-content/uploads/2023/04/cropped-logo-32x32.png devops – Little Big Extra http://littlebigextra.com 32 32 How to convert JSON schema to Java classes using maven plugin http://littlebigextra.com/how-to-convert-json-schema-to-java-classes-using-maven-plugin/ http://littlebigextra.com/how-to-convert-json-schema-to-java-classes-using-maven-plugin/#comments Tue, 09 May 2023 14:59:48 +0000 http://littlebigextra.com/?p=971 Convert JSON schema to Java classes using maven plugin Introduction We often need to convert JSON schema’s to Java classes, there is a jsonschema2pojo-maven-plugin which helps in generating POJO(plain old java objects) from JSON or JSON schema. To use it all we need is add this plugin, update dependencies and run mvn generate Add jsonschema2pojo-maven-plugin […]

The post How to convert JSON schema to Java classes using maven plugin appeared first on Little Big Extra.

]]>
Convert JSON schema to Java classes using maven plugin

Introduction

We often need to convert JSON schema’s to Java classes, there is a jsonschema2pojo-maven-plugin which helps in generating POJO(plain old java objects) from JSON or JSON schema.

To use it all we need is add this plugin, update dependencies and run mvn generate

Add jsonschema2pojo-maven-plugin in POM.XML

Add this plugin to your POM.XML under plugin section

<plugin>
				<groupId>org.jsonschema2pojo</groupId>
				<artifactId>jsonschema2pojo-maven-plugin</artifactId>
				<version>0.4.34</version>
				<configuration>
					<sourceDirectory>${basedir}/src/main/resources/schema</sourceDirectory>
					<targetPackage>com.test.gen</targetPackage>
					<useCommonsLang3>true</useCommonsLang3>
				</configuration>
				<executions>
					<execution>
						<goals>
							<goal>generate</goal>
						</goals>
					</execution>
				</executions>
			</plugin>

Add Maven dependencies

The generated types depend on Commons Lang library for equals, hashCode and toString. Also, you need to add Jackson-databind which contains the data-binding functionality and tree-model for Jackson Data Processor, this jar has a transitive dependency on jackson-core and jackson-annotations so they will be downloaded automatically.

<dependency>
			<groupId>org.apache.commons</groupId>
			<artifactId>commons-lang3</artifactId>
			<version>3.0</version>
		</dependency>
		
		<dependency>
			<groupId>com.fasterxml.jackson.core</groupId>
			<artifactId>jackson-databind</artifactId>
		</dependency>

 

Run the Maven Goal

You can run

mvn generate-sources
goal to generate the java classes. All files will be added in the target/java-gen folder by default
If you need it to be generated in another folder you can make use of
${basedir}/src/main/gen

however I have noticed that specifying any other folder doesn’t add that folder automatically as source folder in eclipse, so you might see some compilation errors.

Make sure that generated folder is added as a source folder in your IDE (Properties ->Java build Path -> Add source folder in eclipse).

Generating classes from JSON and not from schema

If you want to generate Java classes from JSON rather than JSON schema add

<sourceType>json</sourceType>

inside the configuration tag as default source type is JSON Schema.

The post How to convert JSON schema to Java classes using maven plugin appeared first on Little Big Extra.

]]>
http://littlebigextra.com/how-to-convert-json-schema-to-java-classes-using-maven-plugin/feed/ 4
Using Jenkins to Build and Deploy Docker images http://littlebigextra.com/using-jenkins-to-build-and-deploy-docker-images/ http://littlebigextra.com/using-jenkins-to-build-and-deploy-docker-images/#respond Fri, 28 Apr 2023 16:06:19 +0000 http://littlebigextra.com/?p=933 How to use Jenkins to Build and Deploy docker images without Jenkins Docker plugin Introduction Jenkins is one of the most popular continuous integration tools and is widely used. It can be used to create pipelines, run jobs and do automated deployments. There are many plugins available which can be added to Jenkins and makes it […]

The post Using Jenkins to Build and Deploy Docker images appeared first on Little Big Extra.

]]>
How to use Jenkins to Build and Deploy docker images without Jenkins Docker plugin

Introduction

Jenkins is one of the most popular continuous integration tools and is widely used. It can be used to create pipelines, run jobs and do automated deployments.
There are many plugins available which can be added to Jenkins and makes it really powerful.

Recently I wanted to do an automated deployment of Docker Images to docker server and tried using docker-plugin but after spending some time it looked to me that if it asking for too much of information and for each operation you need to provide arguments, I would prefer a solution which is more dynamic and picks things automatically with user providing the bare essentials parameters arguments.
This is how it looks like

 

In a nutshell, all I wanted was

  • Jenkins to automatically start building the Docker Image once the code is pushed to GIT.
  • The Image should be deployed to remote docker host and it should start the container

How to do it

  • Well, the first step is is to enable Jenkins to auto build with BitBucket server or other code repository server. If you want your build process to start as soon as the code is committed to GIT repository.
    Here is an example of how to enable with BitBucket Server, for other GIT repositories servers like GITHUB Jenkins does have different plugins.In case you want to build process to start manually or periodically you can skip this step.

  • Once the build is automated.The second step is to enable Remote API on Docker Host.Docker provides remote REST API which is beneficial if you want to connect to a remote docker host, this API allows you do to set of operations from creating and managing containers to start to stop containers.This step is pivotal before we move on to next step.
  • The third step is to modify your pom to build and deploy Docker Image, well this is the key step as in this step you will be literally doing all the stuff from creating the image to build the image and start the container.

    Once all 3 steps have been tested and completed all is left to be done in Jenkins is to Invoke the clean install goal as shown below.

Jenkins Build step

If you want to push the docker image to some repository after building, testing and deploying.Please follow this link

The post Using Jenkins to Build and Deploy Docker images appeared first on Little Big Extra.

]]> http://littlebigextra.com/using-jenkins-to-build-and-deploy-docker-images/feed/ 0 How to Enable Docker Remote REST API on Docker Host http://littlebigextra.com/how-to-enable-remote-rest-api-on-docker-host/ http://littlebigextra.com/how-to-enable-remote-rest-api-on-docker-host/#comments Sat, 25 Mar 2023 19:23:30 +0000 http://littlebigextra.com/?p=679 Enable Docker Remote REST API on Docker Host in Ubuntu   Introduction Docker provides remote REST API which is beneficial if you want to connect to a remote docker host. Few of the functions which you can achieve using Docker REST API over a simple browser are Create and Manage Containers Get low-level information about a […]

The post How to Enable Docker Remote REST API on Docker Host appeared first on Little Big Extra.

]]>
Enable Docker Remote REST API on Docker Host in Ubuntu

 

Introduction

Docker provides remote REST API which is beneficial if you want to connect to a remote docker host. Few of the functions which you can achieve using Docker REST API over a simple browser are

  • Create and Manage Containers
  • Get low-level information about a container
  • Get Container Logs
  • Start/Stop container
  • Kill a container
My remote docker host was an Ubuntu Virtual Image on Microsoft Azure.

In this tutorial, I will show you

  • What didn’t worked?
  • What really worked.

Things that didn’t worked

Over the internet, most of the people have suggested editing DOCKER_OPTS variable.

  • I changed DOCKER_OPTS in the
    /etc/default/docker
    file but it didn’t have any effect
  • Then I tried changing DOCKER_OPTS in the file
    /etc/init/docker.conf
    but again no success.

What really worked for me to enable docker remote API on docker host

  • Navigate to /lib/system/system in your terminal and open docker.service file
    vi /lib/systemd/system/docker.service
  • Find the line which starts with ExecStart and adds  -H=tcp://0.0.0.0:2375 to make it look like
    ExecStart=/usr/bin/docker daemon -H=fd:// -H=tcp://0.0.0.0:2375
  • Save the Modified File
  • Reload the docker daemon
    systemctl daemon-reload
  • Restart the container
    sudo service docker restart
  • Test if it is working by using this command, if everything is fine below command should return  a JSON
    curl http://localhost:2375/images/json
  • To test remotely, use the PC name or IP address of Docker Host
Please note that docker standard TLS port is 2376 and 2375 are the standard unencrypted port. As we have defined 0.0.0.0, which defines an open interface for everyone. Anyone with network access to this port will have full root access on the host. I have restricted the access by creating an access-controlled list in Azure so please follow some similar mechanism to restrict access

The post How to Enable Docker Remote REST API on Docker Host appeared first on Little Big Extra.

]]>
http://littlebigextra.com/how-to-enable-remote-rest-api-on-docker-host/feed/ 2
How To Push Docker Images To Docker Hub Repository Using Docker Maven plugin http://littlebigextra.com/push-docker-images-docker-hub-using-maven/ http://littlebigextra.com/push-docker-images-docker-hub-using-maven/#comments Sat, 25 Mar 2023 12:15:07 +0000 http://littlebigextra.com/?p=501 Push a Docker image to DockerHub using docker maven plugin fabric8.io   Introduction If you want to push a docker image to Docker Hub repository, it can be achieved using docker maven plugin from fabric8.io. This plugin lets you build images, start and stop containers and push it to Docker repositories In case you are wondering […]

The post How To Push Docker Images To Docker Hub Repository Using Docker Maven plugin appeared first on Little Big Extra.

]]>
Push a Docker image to DockerHub using docker maven plugin fabric8.io

 

Introduction

If you want to push a docker image to Docker Hub repository, it can be achieved using docker maven plugin from fabric8.io. This plugin lets you build images, start and stop containers and push it to Docker repositories

In case you are wondering the difference between a container and image, please note that in docker terminology a running image is called container.
In this tutorial, we are going to

  • Build a docker image
  • Push image to Docker Hub repository

&nsbp;

Want to build a docker image first using this plugin ?
Read Here : How to Build a Docker Image and start stop container
  1. For the below example, I have used Spring Boot Rest service project which is connected to MongoDB. See below link for reference.
  2. Using io.fabric8 plugin, under the plugin section add the io.fabric8 plugin
    <plugin>
    	<groupId>io.fabric8</groupId>
    	<artifactId>docker-maven-plugin</artifactId>
    	<version>0.20.0</version>....

  3. We need to first build the image from Dockerfile, this plugin supports all the command which are in Dockerfile and you can directly create a docker image by giving arguments in pom.xml but In my view making docker images from Docker file is much easier and simpler. So we will ask maven plugin in below step to look for Dockerfile in root( project base directory)
    <name>springboot-mongo-dockerimage:${project.version}</name>
        <alias>springboot-mongo-dockerimage</alias>
    <build>
        <dockerFileDir>${project.basedir}</dockerFileDir>
    </build>
  4. Next we specify the docker registry address
    <registry>registry.hub.docker.com/YOUR_DOCKER_HUB_USERNAME</registry>
  5. Now we need to add authentication credentials in <authconfig> tag
    <authConfig>
    	<username>ENTER YOUR DOCKER HUB USERNAME LIKE abhishek</username>
    	<password>ENTER YOUR DOCKER HUB PASSWORD</password>
    </authConfig>
  6. We will add execution phases, where we will push the image
    <execution>
    	<id>push</id>
    	<phase>post-integration-test</phase>
    	<goals>
    		<goal>push</goal>
    	</goals>
    </execution>
  7. Now run the pom.xml using mvn clean install and check your docker hub repo after some time to see if image has been pushed. If you are pushing the image for first time its gonna take some time  depending on the size of image.

Here is the complete build section from the plugin, hope this help.

<build>
		<plugins>
			<plugin>
				<groupId>org.springframework.boot</groupId>
				<artifactId>spring-boot-maven-plugin</artifactId>
			</plugin>
			<plugin>
				<groupId>io.fabric8</groupId>
				<artifactId>docker-maven-plugin</artifactId>
				<version>0.20.0</version>
				<configuration>
					<registry>registry.hub.docker.com/YOUR_DOCKER_HUB_USERNAME</registry>

					<images>
						<image>
							<name>springboot-mongo-dockerhub:${project.version}</name>
							<alias>springboot-mongo-dockerhub</alias>
							<build>
								<dockerFileDir>${project.basedir}</dockerFileDir>

							</build>
							<run>
								<namingStrategy>alias</namingStrategy>
								<dependsOn>
									<container>mongo</container>
								</dependsOn>

								<links>
									<link>mongo</link>
								</links>
								<ports>
									<port>9876:8080</port>
								</ports>
								<log>
									<prefix>TC</prefix>
									<date>default</date>
									<color>cyan</color>
								</log>
							</run>
						</image>
					</images>
					<authConfig>
						<username>ENTER YOUR DOCKER HUB USERNAME LIKE abhishek</username>
						<password>ENTER YOUR DOCKER HUB PASSWORD</password>
					</authConfig>
				</configuration>

				<executions>
					<execution>
						<id>start</id>
						<phase>pre-integration-test</phase>
						<goals>
							<goal>stop</goal>
							<goal>build</goal>
							<goal>start</goal>
						</goals>
					</execution>
					<execution>
						<id>push</id>
						<phase>post-integration-test</phase>
						<goals>
							<goal>stop</goal>
							<goal>push</goal>
						</goals>
					</execution>
				</executions>
			</plugin>
		</plugins>
	</build>

 

 

If you get an error : io.fabric8:docker-maven-plugin:0.20.0:build failed: A tar file cannot include itself

How to fix- A tar file cannot include itself

The post How To Push Docker Images To Docker Hub Repository Using Docker Maven plugin appeared first on Little Big Extra.

]]>
http://littlebigextra.com/push-docker-images-docker-hub-using-maven/feed/ 2
How to enable jenkins to auto build with BitBucket server http://littlebigextra.com/how-to-enable-jenkins-to-auto-build-with-bit-bicket-server/ http://littlebigextra.com/how-to-enable-jenkins-to-auto-build-with-bit-bicket-server/#respond Fri, 03 Mar 2023 12:35:33 +0000 http://52.32.180.172/?p=406  Integrate BitBucket webhook with Jenkins to auto build on push of code to a bitbucket repository One of the goals of Continuous Integration andContinuous deployment is that once the code is pushed to the repository it should be tested, built and then packaged for deployment. BitBucket webhook makes is possible to trigger real-time notifications to Jenkins […]

The post How to enable jenkins to auto build with BitBucket server appeared first on Little Big Extra.

]]>
 Integrate BitBucket webhook with Jenkins to auto build on push of code to a bitbucket repository

One of the goals of Continuous Integration andContinuous deployment is that once the code is pushed to the repository it should be tested, built and then packaged for deployment. BitBucket webhook makes is possible to trigger real-time notifications to Jenkins server so the auto build starts as soon as the code is pushed.

In this tutorial, we will

  • Add Jenkins add-on/plugin to BitBucket server
  • Install BitBucket Plugin in Jenkins
  • Configure Jenkins
  • Configure BitBucket

Add Jenkins add on to BitBucket server

Once you have installed BitBucket server on your machine, make sure that it is working. The default port is 7990, so you should be able to access it by using localhost: 7990.

  • Click on Settings and on the left side under ADD-ONS, click on Find new add-ons.
  • Search Jenkins and add the plugin

Install BitBucket Plugin in Jenkins

  • Make sure Jenkins is up and Running, in my case, it is running on localhost: 8080
  • In Jenkins go to Jenkins->Plugin Manager and install BitBucket Plugin and then restart Jenkins.

 enable jenkins to auto build with BitBucket server

Configure Jenkins

  • Now go the Job, which you have created and under Source Code Management Plugin, give the GIT Clone URL as shown below in the screenshot.
    BitBucket URL

 

  •  Under Build, Triggers make sure two boxes are checked
    a. Build when a change is published to BitBucket
    b. Poll SCM

build triggers

Configure BitBucket

  • Now go to BitBucket -> Repositories and select the repo for which are Jenkins has been configured

configure BitBucket

  • Click on the setting (icon on the left-hand side) and then hooks. Edit the Bitbucket Server Webhook to Jenkins
      a. Enter your Jenkins URL
      b. Repo Clone URL: Select the URL and the same protocol which BitBucket is providing you to clone. In case username is missing from the URL, we need to add it manually.I had to use Chrome Developer tools and remove the disabled flag from the input box and then add username manually.
    Ignore above step if the username is in URL
        c. Once it is enabled edit the URL with http://UserName@Server/gitrepo

CustomURL

  • Click on Trigger Jenkins if it says successful everything is fine.
    BitBucket server

 

  •  Commit some code and Jenkins should automatically start the build. Refresh the Jenkins page is auto refresh is disabled

The post How to enable jenkins to auto build with BitBucket server appeared first on Little Big Extra.

]]>
http://littlebigextra.com/how-to-enable-jenkins-to-auto-build-with-bit-bicket-server/feed/ 0
How to Install Jenkins on Docker http://littlebigextra.com/how-to-install-jenkins-on-docker/ http://littlebigextra.com/how-to-install-jenkins-on-docker/#respond Tue, 28 Feb 2023 12:30:11 +0000 http://52.37.230.57/?p=266 Installing Docker on Jenkins Jenkins is a great tool to build and package artifacts, it can be either deployed independently as a service in docker, lets go through the steps required to install Jenkins on docker as a container. In this tutorial, we will Get Jenkins Docker Image from Docker Hub Run the Docker Image […]

The post How to Install Jenkins on Docker appeared first on Little Big Extra.

]]>
Installing Docker on Jenkins

Jenkins is a great tool to build and package artifacts, it can be either deployed independently as a service in docker, lets go through the steps required to install Jenkins on docker as a container.

In this tutorial, we will

    • Get Jenkins Docker Image from Docker Hub
    • Run the Docker Image
    • Configure Jenkins

Get official Jenkins Image

      • Go to your terminal/command prompt and get the Jenkins latest’s image from docker hub
        docker pull Jenkins

Run Docker Image

Map localhost ports

        • Then we will try to run the image, we will map Jenkins on localhost port 8080 and port 50000
Port 8080 exposes the web interface and port 50000 gives you access to a remote Java (JIRA) API.

Map Volumes to persist data

      • If it is important to persist the Jenkins data you can choose between the 2 options.
        1. Save data within the docker container, the docker volume persists upon restarts and even after if the container is deleted.
        2. Save data on your host/disk/laptop/machine. Weigh in your requirements and see what fits in with your requirements.
      • Option-1> To persist the data within docker container
        docker volume create --name jenkins_data
        docker run --name jenkins -d -v jenkins_data:/var/jenkins_home -p 8080:8080 -p 50000:50000 jenkins:latest

OR

      • Option-2>To persist the data within host( host machine/laptop) container(option-2)
        docker run --name jenkins -d -v /Users/jenkins:/var/jenkins_home -p 8080:8080 -p 50000:50000 jenkins:latest

        Don’t forget to change the path to your local drive  after -v tag /Users/…. in case of Mac and C:/Users…. in case of Windows)

Check Jenkins is running

        • Check if docker container Jenkins is running
          docker ps -a

          If all is fine, it should show output like below
          CONTAINER    ID    IMAGE    COMMAND    CREATED   STATUS   PORTS    NAMES
          041465f0254e jenkins:latest "/bin/tini -- /usr..." 43 hours ago Up 58 seconds 0.0.0.0:50000>50000/tcp, 0.0.0.0:8082->8080/tcp jenkins

Configure Jenkins

        • Go to Jenkins -> Globals tools configurations and add Java/GIT/Maven installations; download from the internet is an option which you can choose if there are no available installations on your disk as shown below.

Manage Jenkins

Add Maven and JDK

The post How to Install Jenkins on Docker appeared first on Little Big Extra.

]]>
http://littlebigextra.com/how-to-install-jenkins-on-docker/feed/ 0
How to Install Artifactory on Docker http://littlebigextra.com/how-to-install-artifactory-on-docker/ http://littlebigextra.com/how-to-install-artifactory-on-docker/#respond Mon, 27 Feb 2023 16:13:23 +0000 http://52.37.230.57/?p=262 Installing Artifactory on Docker JFrog Artifactory is a repository manager used for versioning deployable like a jar, war, docker image etc. Artifactory Pro is a licensed/paid version which will let you create repositories for docker images and much more.Artifactory OSS is open source and a restrective version where you can create maven repositories and few others. […]

The post How to Install Artifactory on Docker appeared first on Little Big Extra.

]]>
Installing Artifactory on Docker

JFrog Artifactory is a repository manager used for versioning deployable like a jar, war, docker image etc. Artifactory Pro is a licensed/paid version which will let you create repositories for docker images and much more.Artifactory OSS is open source and a restrective version where you can create maven repositories and few others.

In this tutorial, we will

    • Get Artifactory Docker Image from Docker Hub
    • Run the Docker Image
    • Verify Installation

Get official Artifactory Image

Pull the Artifactory OSS Docker Image, it’s an open source image. For this tutorial, we will go ahead with Artifactory OSS

docker pull docker.bintray.io/jfrog/artifactory-oss:latest

 

Run Docker Image

 

Map localhost ports

  • Then we will try to run the image, we will map Docker on localhost port 8081

docker run --name artifactory -d -p 8081:8081 docker.bintray.io/jfrog/artifactory-pro:latest

 

Map Volumes to persist data

It is important to persist the artifacts which will be fed to Artifactory, so we have two options here

    • If it is important to persist the Jenkins data you can choose between the 2 options.
      1. Save data within the docker container, the docker volume persists upon restarts and even after if the container is deleted.
      2. Save data on your host/disk/laptop/machine. Weigh in your requirements and see what fits in with your requirements.

If artifactory was running using above command already then you need to stop the container using

docker stop artifactory
docker rm artifactory

  • Option-1> To persist the data within docker container
    docker volume create --name artifactory5_data
    
    docker run --name artifactory-5.0.0 -d -v artifactory5_data:/var/opt/jfrog/artifactory -p 8081:8081 docker.bintray.io/jfrog/artifactory-pro:latest

    OR

  • Option-2> To persist the data within host( host machine/laptop) container
    docker run --name artifactory-5.0.0 -d -v  /Users/abhishek/jfrog/artifactory:/var/opt/jfrog/artifactory -p 8081:8081 docker.bintray.io/jfrog/artifactory-oss:latest
Don’t forget to change the path to your local drive  after -v tag /Users/…. in case of Mac and C:/Users…. in case of Windows

Verify Installation

  • Since Artifactory was mapped to localhost port 8081, open your browser and use localhost:8081 if accessing from the same machine.

http://SERVER_DOMAIN:8081/artifactory

The post How to Install Artifactory on Docker appeared first on Little Big Extra.

]]>
http://littlebigextra.com/how-to-install-artifactory-on-docker/feed/ 0
How to add remote archetype catalog in Eclipse http://littlebigextra.com/how-to-add-remote-archetype-catalog-in-eclipse/ http://littlebigextra.com/how-to-add-remote-archetype-catalog-in-eclipse/#respond Fri, 24 Feb 2023 17:03:27 +0000 http://52.37.230.57/?p=252 Add Maven Remote catalogue in eclipse Maven remote catalogues are very useful in starting a project quickly and neatly. The project structure including src, test directory are created along with pom dependencies. Follow below steps to add remote archetype catalogue Go to Windows -> Preferences -> Maven ->Archetype Click on Add Remote Catalog Add catalogue […]

The post How to add remote archetype catalog in Eclipse appeared first on Little Big Extra.

]]>
Add Maven Remote catalogue in eclipse

Maven remote catalogues are very useful in starting a project quickly and neatly. The project structure including src, test directory are created along with pom dependencies.

Follow below steps to add remote archetype catalogue

  1. Go to Windows -> Preferences -> Maven ->Archetype
  2. Click on Add Remote Catalog
    Remote Maven archetype
  3. Add catalogue file as
    • http://repo1.maven.org/maven2/archetype-catalog.xml
  4. Description as
    • Remote Archetype catalogue
  5. Click on Verify, you should see a message like Found XXX archetype’s

If you are behind a proxy you might need to go to Windows -> Preference-> Network and add the proxy details

Why not use this remote archetype to create your first Spring Boot Project

Follow this Video for more details

The post How to add remote archetype catalog in Eclipse appeared first on Little Big Extra.

]]>
http://littlebigextra.com/how-to-add-remote-archetype-catalog-in-eclipse/feed/ 0