Python Project - Deep Learning - Colorize Black and White Images with Python
Posted by Superadmin on August 22 2020 14:42:58

Python Project - Deep Learning - Colorize Black and  White Images with Python

BY  · UPDATED · MAY 21, 2020

 

This Deep Learning Project aims to provide colorizing black & white images with Python.

In image colorization, we take a black and white image as input and produce a colored image. We will solve this project with OpenCV deep neural network.

Deep Learning Project Colorize Images Python

 

Deep Learning Project – Colorize Black & White Images with Python


Lab Color Space:

Like RGB, Lab is another color space. It is also three channel color space like RGB where the channels are:

In this color space, the grayscale part of the image is only encoded in L channel. Therefore Lab color space is more favorable for our project.

 

Problem Statement:

deep learning project colorize black white images with python

We can formulate our problem statement as to predict a and b channels, given an input grayscale image.

In this deep learning project, we will use OpenCV DNN architecture which is trained on ImageNet dataset. The neural net is trained with the L channel of images as input data and a,b channels as target data.

Steps to implement Image Colorization Project:

For colorizing black and white images we will be using a pre-trained caffe model , a prototxt file, and a NumPy file.

The prototxt file defines the network and the numpy file stores the cluster center points in numpy format.

1. Make a directory with name models.

  1. mkdir models

2. Open the terminal and run following commands to download the caffemodel, prototxt file and the NumPy file.

  1. wget https://github.com/richzhang/colorization/blob/master/colorization/resources/pts_in_hull.npy?raw=true -O ./pts_in_hull.npy
  3. wget https://raw.githubusercontent.com/richzhang/colorization/master/colorization/models/colorization_deploy_v2.prototxt -O ./models/colorization_deploy_v2.prototxt
  4. wget http://eecs.berkeley.edu/~rich.zhang/projects/2016_colorization/files/demo_v2/colorization_release_v2.caffemodel -O ./models/colorization_release_v2.caffemodel

3. Create a python file image_colorization.py and paste the code specified in the below steps

4. Imports:

  1. import numpy as np
  2. import cv2 as cv
  3. import os.path

5. Read B&W image and load the caffemodel:

  1. frame = cv.imread(“__test_image_path__”)
  2. numpy_file = np.load('./pts_in_hull.npy')
  3. Caffe_net = cv.dnn.readNetFromCaffe("./models/colorization_deploy_v2.prototxt", "./models/colorization_release_v2.caffemodel")

6. Add layers to the caffe model:

  1. numpy_file = numpy_file.transpose().reshape(2, 313, 1, 1)
  2. Caffe_net.getLayer(Caffe_net.getLayerId('class8_ab')).blobs = [numpy_file.astype(np.float32)]
  3. Caffe_net.getLayer(Caffe_net.getLayerId('conv8_313_rh')).blobs = [np.full([1, 313], 2.606, np.float32)]

7. Extract L channel and resize it:

  1. input_width = 224
  2. input_height = 224
  3. rgb_img = (frame[:,:,[2, 1, 0]] * 1.0 / 255).astype(np.float32)
  4. lab_img = cv.cvtColor(rgb_img, cv.COLOR_RGB2Lab)
  5. l_channel = lab_img[:,:,0]
  6. l_channel_resize = cv.resize(l_channel, (input_width, input_height))
  7. l_channel_resize -= 50

8. Predict the ab channel and save the result:

  1. Caffe_net.setInput(cv.dnn.blobFromImage(l_channel_resize))
  2. ab_channel = Caffe_net.forward()[0,:,:,:].transpose((1,2,0))
  3. (original_height,original_width) = rgb_img.shape[:2]
  4. ab_channel_us = cv.resize(ab_channel, (original_width, original_height))
  5. lab_output = np.concatenate((l_channel[:,:,np.newaxis],ab_channel_us),axis=2)
  6. bgr_output = np.clip(cv.cvtColor(lab_output, cv.COLOR_Lab2BGR), 0, 1)
  7. cv.imwrite("./result.png", (bgr_output*255).astype(np.uint8))

Results:

Now execute the deep learning project – run the python file with path of a grayscale image to test our results.

  1. python3 image_colorization.py

After running the above python file, the colorized image will be saved in your working directory as result.png

Deep Learning Project Colorize Images Python

Code for GUI:

Make a new python file gui.py in the present directory. Paste the below code for image colorization interface.

  1. import tkinter as tk
  2. from tkinter import *
  3. from tkinter import filedialog
  4. from PIL import Image, ImageTk
  5. import os
  6. import numpy as np
  7. import cv2 as cv
  8. import os.path
  9. numpy_file = np.load('./pts_in_hull.npy')
  10. Caffe_net = cv.dnn.readNetFromCaffe("./models/colorization_deploy_v2.prototxt", "./models/colorization_release_v2.caffemodel")
  11. numpy_file = numpy_file.transpose().reshape(2, 313, 1, 1)
  12. class Window(Frame):
  13. def __init__(self, master=None):
  14. Frame.__init__(self, master)
  15. self.master = master
  16. self.pos = []
  17. self.master.title("B&W Image Colorization")
  18. self.pack(fill=BOTH, expand=1)
  19. menu = Menu(self.master)
  20. self.master.config(menu=menu)
  21. file = Menu(menu)
  22. file.add_command(label="Upload Image", command=self.uploadImage)
  23. file.add_command(label="Color Image", command=self.color)
  24. menu.add_cascade(label="File", menu=file)
  25. self.canvas = tk.Canvas(self)
  26. self.canvas.pack(fill=tk.BOTH, expand=True)
  27. self.image = None
  28. self.image2 = None
  29. label1=Label(self,image=img)
  30. label1.image=img
  31. label1.place(x=400,y=370)
  32. def uploadImage(self):
  33. filename = filedialog.askopenfilename(initialdir=os.getcwd())
  34. if not filename:
  35. return
  36. load = Image.open(filename)
  37. load = load.resize((480, 360), Image.ANTIALIAS)
  38. if self.image is None:
  39. w, h = load.size
  40. width, height = root.winfo_width(), root.winfo_height()
  41. self.render = ImageTk.PhotoImage(load)
  42. self.image = self.canvas.create_image((w / 2, h / 2), image=self.render)
  44. else:
  45. self.canvas.delete(self.image3)
  46. w, h = load.size
  47. width, height = root.winfo_screenmmwidth(), root.winfo_screenheight()
  49. self.render2 = ImageTk.PhotoImage(load)
  50. self.image2 = self.canvas.create_image((w / 2, h / 2), image=self.render2)
  51. frame = cv.imread(filename)
  53. Caffe_net.getLayer(Caffe_net.getLayerId('class8_ab')).blobs = [numpy_file.astype(np.float32)]
  54. Caffe_net.getLayer(Caffe_net.getLayerId('conv8_313_rh')).blobs = [np.full([1, 313], 2.606, np.float32)]
  55. input_width = 224
  56. input_height = 224
  57. rgb_img = (frame[:,:,[2, 1, 0]] * 1.0 / 255).astype(np.float32)
  58. lab_img = cv.cvtColor(rgb_img, cv.COLOR_RGB2Lab)
  59. l_channel = lab_img[:,:,0]
  60. l_channel_resize = cv.resize(l_channel, (input_width, input_height))
  61. l_channel_resize -= 50
  62. Caffe_net.setInput(cv.dnn.blobFromImage(l_channel_resize))
  63. ab_channel = Caffe_net.forward()[0,:,:,:].transpose((1,2,0))
  64. (original_height,original_width) = rgb_img.shape[:2]
  65. ab_channel_us = cv.resize(ab_channel, (original_width, original_height))
  66. lab_output = np.concatenate((l_channel[:,:,np.newaxis],ab_channel_us),axis=2)
  67. bgr_output = np.clip(cv.cvtColor(lab_output, cv.COLOR_Lab2BGR), 0, 1)
  69. cv.imwrite("./result.png", (bgr_output*255).astype(np.uint8))
  70. def color(self):
  71. load = Image.open("./result.png")
  72. load = load.resize((480, 360), Image.ANTIALIAS)
  73. if self.image is None:
  74. w, h = load.size
  75. self.render = ImageTk.PhotoImage(load)
  76. self.image = self.canvas.create_image((w / 2, h/2), image=self.render)
  77. root.geometry("%dx%d" % (w, h))
  78. else:
  79. w, h = load.size
  80. width, height = root.winfo_screenmmwidth(), root.winfo_screenheight()
  81. self.render3 = ImageTk.PhotoImage(load)
  82. self.image3 = self.canvas.create_image((w / 2, h / 2), image=self.render3)
  83. self.canvas.move(self.image3, 500, 0)
  85. root = tk.Tk()
  86. root.geometry("%dx%d" % (980, 600))
  87. root.title("B&W Image Colorization GUI")
  88. img = ImageTk.PhotoImage(Image.open("logo2.png"))
  89. app = Window(root)
  90. app.pack(fill=tk.BOTH, expand=1)
  91. root.mainloop()

Summary:

This tutorial guides you how to build a deep learning project to colorize black and white images. It first introduces you to Lab color space and why it is favorable for our problem statement. Then step by step it describes how to implement black and white image colorizer.