Machine Learning Integeration with jenkins+docker+git .

60% of Machine Learning Projects are never implemented.??

Investor are willing to invest in cutting edge projects with the power to revolutionize their organization, but they struggle to operationalize these projects. Approximately 40 percent reach implementation, while 60 percent stall or flame out.

Here i am make a project which is integeration of machine learning and dev operation.

1. Create a container image that has Python3 and Keras or NumPy installed using dockerfile 2. When we launch this image, it should automatically start to train the model in the container.

3. Create a job chain of job1, job2, job3, job4, and job5 using the build pipeline plugin in Jenkins

4. Job1: Pull the Github repo automatically when some developers push the repo to Github.

5. Job2: By looking at the code or program file, Jenkins should automatically start the respective machine learning software installed interpreter install image container to deploy code and start training( eg. If code uses CNN, then Jenkins should start the container that has already installed all the software required for the CNN processing).

6. Job3: In this first, it checks the accuracy then take action accordingly if accuracy is greater than expected then it send mail otherwise it launch the container for tweaking

8. Job4: send slack notification if expected accuracy achieved

Step1: Create docker file

FROM centos
RUN pip3 install — upgrade pip
RUN pip3 install tensorflow
RUN pip3 install keras
RUN pip3 install pillow
RUN pip3 install opencv-python
RUN pip3 install numpy

RUN pip3 install pandas

ENTRYPOINT [“python36”]

WORKDIR /home/ashutosh/Documents/mlops/
COPY mnsit_keras_library_cnn.py /home/ashutosh/Documents/mlops/

CMD [“python36”,”mnsit_keras_library_cnn.py”]

step2- Build Docker image

when i run this container it autmatic train the model

step3-Pull the Github repo automatically when some developers push the repo to Github and copy the downloaded

Step3: By looking at the code or program file, Jenkins should automatically start the respective machine learning software installed docker image container to deploy code and start training.

step4- check accuarcy of model if it meet require accuarcy then it send notification otherwise it start new container which run tweak.py file which add more hidden layer in CNN network

Step5- if all condition are satisfy and accuarcy is greater than 92 then it send notification on slack channel

for reference here is my git hub repo.

https://github.com/ashut08/MLops.git