Tensorflow Convolutional Neural Networks | Convolutional Neural Networks

This tutorial demonstrates training a simple Convolutional Neural Network (CNN) to classify CIFAR images. Because this tutorial uses the Keras Sequential API, creating and training your model will take just a few lines of code.

Import TensorFlow


import tensorflow as tf from tensorflow.keras import datasets, layers, models import matplotlib.pyplot as plt


2023-10-27 06:01:15.153603: E external/local_xla/xla/stream_executor/cuda/cuda_dnn.cc:9261] Unable to register cuDNN factory: Attempting to register factory for plugin cuDNN when one has already been registered 2023-10-27 06:01:15.153656: E external/local_xla/xla/stream_executor/cuda/cuda_fft.cc:607] Unable to register cuFFT factory: Attempting to register factory for plugin cuFFT when one has already been registered 2023-10-27 06:01:15.155401: E external/local_xla/xla/stream_executor/cuda/cuda_blas.cc:1515] Unable to register cuBLAS factory: Attempting to register factory for plugin cuBLAS when one has already been registered

Download and prepare the CIFAR10 dataset

The CIFAR10 dataset contains 60,000 color images in 10 classes, with 6,000 images in each class. The dataset is divided into 50,000 training images and 10,000 testing images. The classes are mutually exclusive and there is no overlap between them.


(train_images, train_labels), (test_images, test_labels) = datasets.cifar10.load_data() # Normalize pixel values to be between 0 and 1 train_images, test_images = train_images / 255.0, test_images / 255.0


Downloading data from https://www.cs.toronto.edu/~kriz/cifar-10-python.tar.gz 170498071/170498071 [==============================] – 2s 0us/step

Verify the data

To verify that the dataset looks correct, let’s plot the first 25 images from the training set and display the class name below each image:


class_names = ['airplane', 'automobile', 'bird', 'cat', 'deer', 'dog', 'frog', 'horse', 'ship', 'truck'] plt.figure(figsize=(10,10)) for i in range(25): plt.subplot(5,5,i+1) plt.xticks([]) plt.yticks([]) plt.grid(False) plt.imshow(train_images[i]) # The CIFAR labels happen to be arrays, # which is why you need the extra index plt.xlabel(class_names[train_labels[i][0]]) plt.show()


Create the convolutional base

The 6 lines of code below define the convolutional base using a common pattern: a stack of Conv2D and MaxPooling2D layers.

As input, a CNN takes tensors of shape (image_height, image_width, color_channels), ignoring the batch size. If you are new to these dimensions, color_channels refers to (R,G,B). In this example, you will configure your CNN to process inputs of shape (32, 32, 3), which is the format of CIFAR images. You can do this by passing the argument

input_shape

to your first layer.


model = models.Sequential() model.add(layers.Conv2D(32, (3, 3), activation='relu', input_shape=(32, 32, 3))) model.add(layers.MaxPooling2D((2, 2))) model.add(layers.Conv2D(64, (3, 3), activation='relu')) model.add(layers.MaxPooling2D((2, 2))) model.add(layers.Conv2D(64, (3, 3), activation='relu'))


Let’s display the architecture of your model so far:


model.summary()


Model: “sequential” _________________________________________________________________ Layer (type) Output Shape Param # ================================================================= conv2d (Conv2D) (None, 30, 30, 32) 896 max_pooling2d (MaxPooling2 (None, 15, 15, 32) 0 D) conv2d_1 (Conv2D) (None, 13, 13, 64) 18496 max_pooling2d_1 (MaxPoolin (None, 6, 6, 64) 0 g2D) conv2d_2 (Conv2D) (None, 4, 4, 64) 36928 ================================================================= Total params: 56320 (220.00 KB) Trainable params: 56320 (220.00 KB) Non-trainable params: 0 (0.00 Byte) _________________________________________________________________

Above, you can see that the output of every Conv2D and MaxPooling2D layer is a 3D tensor of shape (height, width, channels). The width and height dimensions tend to shrink as you go deeper in the network. The number of output channels for each Conv2D layer is controlled by the first argument (e.g., 32 or 64). Typically, as the width and height shrink, you can afford (computationally) to add more output channels in each Conv2D layer.

Add Dense layers on top

To complete the model, you will feed the last output tensor from the convolutional base (of shape (4, 4, 64)) into one or more Dense layers to perform classification. Dense layers take vectors as input (which are 1D), while the current output is a 3D tensor. First, you will flatten (or unroll) the 3D output to 1D, then add one or more Dense layers on top. CIFAR has 10 output classes, so you use a final Dense layer with 10 outputs.


model.add(layers.Flatten()) model.add(layers.Dense(64, activation='relu')) model.add(layers.Dense(10))


Here’s the complete architecture of your model:


model.summary()


Model: “sequential” _________________________________________________________________ Layer (type) Output Shape Param # ================================================================= conv2d (Conv2D) (None, 30, 30, 32) 896 max_pooling2d (MaxPooling2 (None, 15, 15, 32) 0 D) conv2d_1 (Conv2D) (None, 13, 13, 64) 18496 max_pooling2d_1 (MaxPoolin (None, 6, 6, 64) 0 g2D) conv2d_2 (Conv2D) (None, 4, 4, 64) 36928 flatten (Flatten) (None, 1024) 0 dense (Dense) (None, 64) 65600 dense_1 (Dense) (None, 10) 650 ================================================================= Total params: 122570 (478.79 KB) Trainable params: 122570 (478.79 KB) Non-trainable params: 0 (0.00 Byte) _________________________________________________________________

The network summary shows that (4, 4, 64) outputs were flattened into vectors of shape (1024) before going through two Dense layers.

Compile and train the model


model.compile(optimizer='adam', loss=tf.keras.losses.SparseCategoricalCrossentropy(from_logits=True), metrics=['accuracy']) history = model.fit(train_images, train_labels, epochs=10, validation_data=(test_images, test_labels))


Epoch 1/10 WARNING: All log messages before absl::InitializeLog() is called are written to STDERR I0000 00:00:1698386490.372362 489369 device_compiler.h:186] Compiled cluster using XLA! This line is logged at most once for the lifetime of the process. 1563/1563 [==============================] – 10s 5ms/step – loss: 1.5211 – accuracy: 0.4429 – val_loss: 1.2497 – val_accuracy: 0.5531 Epoch 2/10 1563/1563 [==============================] – 6s 4ms/step – loss: 1.1408 – accuracy: 0.5974 – val_loss: 1.1474 – val_accuracy: 0.6023 Epoch 3/10 1563/1563 [==============================] – 6s 4ms/step – loss: 0.9862 – accuracy: 0.6538 – val_loss: 0.9759 – val_accuracy: 0.6582 Epoch 4/10 1563/1563 [==============================] – 6s 4ms/step – loss: 0.8929 – accuracy: 0.6879 – val_loss: 0.9412 – val_accuracy: 0.6702 Epoch 5/10 1563/1563 [==============================] – 6s 4ms/step – loss: 0.8183 – accuracy: 0.7131 – val_loss: 0.8830 – val_accuracy: 0.6967 Epoch 6/10 1563/1563 [==============================] – 6s 4ms/step – loss: 0.7588 – accuracy: 0.7334 – val_loss: 0.8671 – val_accuracy: 0.7039 Epoch 7/10 1563/1563 [==============================] – 6s 4ms/step – loss: 0.7126 – accuracy: 0.7518 – val_loss: 0.8972 – val_accuracy: 0.6897 Epoch 8/10 1563/1563 [==============================] – 7s 4ms/step – loss: 0.6655 – accuracy: 0.7661 – val_loss: 0.8412 – val_accuracy: 0.7111 Epoch 9/10 1563/1563 [==============================] – 7s 4ms/step – loss: 0.6205 – accuracy: 0.7851 – val_loss: 0.8581 – val_accuracy: 0.7109 Epoch 10/10 1563/1563 [==============================] – 7s 4ms/step – loss: 0.5872 – accuracy: 0.7937 – val_loss: 0.8817 – val_accuracy: 0.7113

Evaluate the model


plt.plot(history.history['accuracy'], label='accuracy') plt.plot(history.history['val_accuracy'], label = 'val_accuracy') plt.xlabel('Epoch') plt.ylabel('Accuracy') plt.ylim([0.5, 1]) plt.legend(loc='lower right') test_loss, test_acc = model.evaluate(test_images, test_labels, verbose=2)


313/313 – 1s – loss: 0.8817 – accuracy: 0.7113 – 655ms/epoch – 2ms/step


print(test_acc)


0.7113000154495239

Your simple CNN has achieved a test accuracy of over 70%. Not bad for a few lines of code! For another CNN style, check out the TensorFlow 2 quickstart for experts example that uses the Keras subclassing API and

tf.GradientTape

.

Course

Convolutional Neural Networks (CNN) with TensorFlow Tutorial

Imagine being in a zoo trying to recognize if a given animal is a cheetah or a leopard. As a human, your brain can effortlessly analyze body and facial features to come to a valid conclusion. In the same way, Convolutional Neural Networks (CNNs) can be trained to perform the same recognition task, no matter how complex the patterns are. This makes them powerful in the field of computer vision.

This conceptual CNN tutorial will start by providing an overview of what CNNs are and their importance in machine learning. Then it will walk you through a step-by-step implementation of CNN in TensorFlow Framework 2.

There are 4 modules in this course

In the first course in this specialization, you had an introduction to TensorFlow, and how, with its high level APIs you could do basic image classification, and you learned a little bit about Convolutional Neural Networks (ConvNets). In this course you’ll go deeper into using ConvNets will real-world data, and learn about techniques that you can use to improve your ConvNet performance, particularly when doing image classification!In Week 1, this week, you’ll get started by looking at a much larger dataset than you’ve been using thus far: The Cats and Dogs dataset which had been a Kaggle Challenge in image classification!

What’s included

8 videos8 readings1 quiz1 programming assignment

You’ve heard the term overfitting a number of times to this point. Overfitting is simply the concept of being over specialized in training — namely that your model is very good at classifying what it is trained for, but not so good at classifying things that it hasn’t seen. In order to generalize your model more effectively, you will of course need a greater breadth of samples to train it on. That’s not always possible, but a nice potential shortcut to this is Image Augmentation, where you tweak the training set to potentially increase the diversity of subjects it covers. You’ll learn all about that this week!

What’s included

7 videos7 readings1 quiz1 programming assignment

Building models for yourself is great, and can be very powerful. But, as you’ve seen, you can be limited by the data you have on hand. Not everybody has access to massive datasets or the compute power that’s needed to train them effectively. Transfer learning can help solve this — where people with models trained on large datasets train them, so that you can either use them directly, or, you can use the features that they have learned and apply them to your scenario. This is Transfer learning, and you’ll look into that this week!

What’s included

7 videos5 readings1 quiz1 programming assignment

You’ve come a long way, Congratulations! One more thing to do before we move off of ConvNets to the next module, and that’s to go beyond binary classification. Each of the examples you’ve done so far involved classifying one thing or another — horse or human, cat or dog. When moving beyond binary into Categorical classification there are some coding considerations you need to take into account. You’ll look at them this week!

What’s included

6 videos8 readings1 quiz1 programming assignment

Instructor

Offered by

Recommended if you’re interested in Machine Learning

Tensors vs Matrices: Differences

Many people confuse tensors with matrices. Even though these two objects look similar, they have completely different properties. This section provides a better understanding of the difference between matrices and tensors.

• We can think of a matrice as a tensor with only two dimensions.
• Tensors, on the other hand, is a more general format that can have any number of dimensions.

As opposed to matrices, tensors are more suitable for deep learning problems for the following reasons:

• They can deal with any number of dimensions, which makes them a better fit for multi-dimensional data.
• Tensors’ ability to be compatible with a wide range of data types, shapes, and dimensions makes them more versatile than matrices.
• Tensorflow provides GPU and TPU support to speed up computations. Using tensors, machine learning engineers can automatically take advantage of these benefits.
• Tensors natively support broadcasting, which consists of making arithmetic operations between tensors of different shapes, which is not always possible when dealing with matrices.

How Does CNN Recognize Images?

Consider the following images:

The boxes that are colored represent a pixel value of 1, and 0 if not colored.

When you press backslash (\), the below image gets processed.

When you press forward-slash (/), the below image is processed:

Here is another example to depict how CNN recognizes an image:

As you can see from the above diagram, only those values are lit that have a value of 1.

How to easily run CNN with Tensorflow in cnvrg.io

Now, with cnvrg.io you can run this pipeline without configuring the different platforms which makes it much faster and easier to run. Using cnvrg.io, you can easily track training progress and serve the model as a REST endpoint. First, you can spin up a VS Code workspace inside cnvrg.io to build our training script from the notebook code. You can use the exact code and ensure that the model is saved at the end of the training.

Run your code as an experiment

Next, you can launch this training script as an experiment. cnvrg.io will provision resources to execute the script and monitor the performance automatically. Resource and training metrics are automatically visualized along with the logs, and all files that were written to disk during the experiment are saved as artifacts in cnvrg.io’s object store.

Make predictions in a few clicks

Now that you have your model, you’ll need to create a “predict” function. cnvrg.io makes it easy, by automatically wrapping this function into a production-grade Flask application equipped with load balancing, autoscaling, monitoring. This file loads the model into memory and uses it in the predict function, which will format the incoming data and return a prediction.

Deploy your predictions to an endpoint

Next, you’ll want to create an endpoint that routes to that function. You could also specify compute resources and autoscaling configurations here too.

Track and monitor your endpoints

cnvrg.io automatically displays metrics such as the number of requests and latency for the endpoint. It also comes with Grafana and Kibana integrated for increased visibility into model

Finally, if you want to trigger retraining and deploying the model as part of a CI/CD pipeline, cnvrg.io provides Flows. The pipeline could programatically trigger the flow via cnvrg.io’s CLI or SDK.

You can test it out in cnvrg.io now by installing cnvrg.io CORE our free community MLOps platform on your Kuberentes here.

Training the model

The next step is to train the model. In this case, `y` is not passed in. That’s taken care of by the function used to generate the training set. Passing the validation data is critical so that the loss and accuracy can be accessed later and plotted. Let’s also reuse the callbacks that were defined in the last section.

<br /> history = model.fit(training_set,validation_data=validation_set, epochs=600,callbacks=callbacks)<br />

Monitoring the model’s performance

Using the history object, the training losses and accuracies can be obtained.

<br /> import pandas as pd metrics_df = pd.DataFrame(history.history)<br />

You can plot them in order to see the learning curves. Let’s start by comparing the training and validation loss.

<br /> metrics_df[[&#8220;loss&#8221;,&#8221;val_loss&#8221;]].plot();<br />

Next, on to the training and validation accuracy.

<br /> metrics_df[[&#8220;binary_accuracy&#8221;,&#8221;val_binary_accuracy&#8221;]].plot();<br />

Installation

!git clone https://github.com/faizan1234567/Convolutional-Neural-networks-in-TensorFlow cd Convolutional-Neural-networks-in-TensorFlow

create virtual environment

python -m venv –system-site-packages .\CNN_with_tf

Activate the virtual environment

.\CNN_with_tf\Scripts\Activate

Install packages within a virtual environment without affecting the host system setup. Start by upgrading pip:

pip install –upgrade pip pip list # show packages installed within the virtual environment

And to exit the virtual environment later:

deactivate # don’t exit until you’re done using TensorFlow

Now install TensorFlow

pip install –upgrade tensorflow

To check everything works perfectly, run the following command

python -c “import tensorflow as tf;print(tf.reduce_sum(tf.random.normal([1000, 1000])))”

You are all set to use tensorflow now. Please follow the notebooks to import or install modules.

Quick Tutorial: Building a Basic Convolutional Neural Network (CNN) in TensorFlow

This quick tutorial can help you get started implementing CNN in TensorFlow. It is based on the Fashion-MNIST dataset, containing 28 x 28 grayscale images of 65,000 fashion products in 10 categories. There are 55,000 images in the training set and 10,000 images in the test set. Our code is based on the full tutorial by Aditya Sharma.

Loading Data

First import all the necessary modules: NumPy, matplotlib and Tensorflow, then import the Fashion-MNIST data as follows:

# Use this for reading the data/fashion directory from the datasetdata = input_data.read_data_sets(‘data/fashion’,one_hot=True,\# Use this for retrieving Fashion-MNIST dataset from Amazon S3 bucket source_url=’http://fashion-mnist.s3-website.eu-central-1.amazonaws.com/’)

CNN Architecture

We will use three convolutional layers, progressively adding more filters. All the filters are 3×3:

1. Layer with hav 32 filters
2. Layer with 64 filters
3. Layer with 128 filters

In addition, we’ll have three max-pooling layers in between the convolutions, which are 2×2.

We’ll set basic hyperparameters of the CNN model:

training_iters = 10learning_rate = 0.001batch_size = 128

This batch size spec tells TensorFlow to train a specified number of images, and do this for every batch.

Neural Network Parameters

The number of inputs to the CNN is 784, because the images have 784 pixels and are read as a 784 dimensional vector. We will rebuild this vector into a matrix of 28 x 28 x 1.

# Use this to specify 28 inputs, and 10 classes for the predicted label at the endn_input = 28n_classes = 10

Here we define an input placeholder x with dimensionality None x 784, and output placeholder size of None x 10. Similarly, we’ll define a placeholder y for the label of the training images, which will be a None x 10 matrix.

We are setting the “row” to None because we previously defined batch_size, meaning placeholders receive the row size when the training set is loaded. Row size will be set to 128, like the batch_size.

# x is the input placeholder, rebuilding the image into 28x28x1 matrixx = tf.placeholder(“float”, [None, 28,28,1])# y is the label set, using the number of classesy = tf.placeholder(“float”, [None, n_classes])

Wrapper Functions

Because we have several layers of the same type in the model, it’s useful to create a wrapper function for each type of layer, to avoid duplicating code. You can get functions like this out of the box with Keras, which is included with Tensorflow. However, in this tutorial we show you how to do things from scratch in TensorFlow without Keras helper functions.

Here is a function creating a 2-dimensional convolutional layer, with bias and Relu activation. The arguments are the test images x, weights W, bias b, and number of strides, meaning how quickly the filter moves over the image during the convolution.

def conv2d(x, W, b, strides=1):x = tf.nn.conv2d(x, W, strides=[1, strides, strides, 1], padding=’SAME’)x = tf.nn.bias_add(x, b)return tf.nn.relu(x)

Here is another function creating a 2D max-pool layer. Here the parameters are test images x, and k, specifying the kernel/filter size.

def maxpool2d(x, k=2):return tf.nn.max_pool(x, ksize=[1, k, k, 1], strides=[1, k, k, 1],padding=’SAME’)

Now let’s define weights and biases.

weights = {‘wc1’: tf.get_variable(‘W0′, shape=(3,3,1,32), initializer=tf.contrib.layers.xavier_initializer()),’wc2’: tf.get_variable(‘W1′, shape=(3,3,32,64), initializer=tf.contrib.layers.xavier_initializer()),’wc3’: tf.get_variable(‘W2′, shape=(3,3,64,128), initializer=tf.contrib.layers.xavier_initializer()),’wd1’: tf.get_variable(‘W3′, shape=(4*4*128,128), initializer=tf.contrib.layers.xavier_initializer()),’out’: tf.get_variable(‘W6’, shape=(128,n_classes), initializer=tf.contrib.layers.xavier_initializer()),}biases = {‘bc1’: tf.get_variable(‘B0′, shape=(32), initializer=tf.contrib.layers.xavier_initializer()),’bc2’: tf.get_variable(‘B1′, shape=(64), initializer=tf.contrib.layers.xavier_initializer()),’bc3’: tf.get_variable(‘B2′, shape=(128), initializer=tf.contrib.layers.xavier_initializer()),’bd1’: tf.get_variable(‘B3′, shape=(128), initializer=tf.contrib.layers.xavier_initializer()),’out’: tf.get_variable(‘B4’, shape=(10), initializer=tf.contrib.layers.xavier_initializer()),

Building the CNN

Now we build the CNN by feeding the weights and biases into the wrapper functions.

def conv_net(x, weights, biases):

# This constructs the first convolutional layer with 32 3×3 filters and 32 biases. The next specifies the max-pool layer with the kernel size set to 2.

conv1 = conv2d(x, weights[‘wc1’], biases[‘bc1’])conv1 = maxpool2d(conv1, k=2)

# Use this to construct the second convolutional layer with 64 3×3 filters and 64 biases, and to another max-pool layer.

conv2 = conv2d(conv1, weights[‘wc2’], biases[‘bc2’])conv2 = maxpool2d(conv2, k=2)

# This helps you construct the third convolutional layer with 128 3×3 filters and 128 biases, and add the last max-pool layer.

conv3 = conv2d(conv2, weights[‘wc3’], biases[‘bc3’])conv3 = maxpool2d(conv3, k=2)

# Now you need to build the fully connected layer that will generate prediction labels. To do this, use reshape() to adapt the output of pooling to the input expected by the fully connected layer.

fc1 = tf.reshape(conv3, [-1,weights[‘wd1’].get_shape().as_list()[0]])fc1 = tf.add(tf.matmul(fc1, weights[‘wd1’]), biases[‘bd1’])

# In this last part, apply the Relu function and perform matrix multiplication on the weights

fc1 = tf.nn.relu(fc1)out = tf.add(tf.matmul(fc1, weights[‘out’]), biases[‘out’])return out

Loss and Optimizer Nodes

First build the model using the conv_net() function we showed above. Pass in the following:

x, weights, and biases. pred = conv_net(x, weights, biases)

This is a multi-class classification problem, so we will use the softmax activation function, which gives a probability between 0 and 1 for each class label (the label with the highest probability will be the prediction of the model). We’ll use cross-entropy as the loss function.

cost = tf.reduce_mean(tf.nn.softmax_cross_entropy_with_logits(logits=pred, labels=y))

Finally, we’ll define the Adam optimizer with a learning rate of 0.001 as defined in the model hyperparameters above:

optimizer = tf.train.AdamOptimizer(learning_rate=learning_rate).minimize(cost)

Evaluate the Model

To test the model, we first initialize weights and biases, and then define a correct_prediction and accuracy node that will evaluate model performance every time it is run.

init = tf.global_variables_initializer()correct_prediction = tf.equal(tf.argmax(pred, 1), tf.argmax(y, 1))accuracy = tf.reduce_mean(tf.cast(correct_prediction, tf.float32))

Now you can start the computation graph, and run a training session as follows:

• Create For loops that define the number of training iterations as specified above
• Create an inner For loop to specify the number of batches we specified above
• Pass training images and labels using variables batch_x and batch_y
• Define x and y placeholders to hold parameters the training images
• After each training iteration, run the loss function and check training accuracy
• After running through all the images, test accuracy by processing the 10,000 test images

See the original tutorial for the complete code that you can use to run the CNN model.

What is a CNN?

A Convolutional Neural Network (CNN or ConvNet) is a deep learning algorithm specifically designed for any task where object recognition is crucial such as image classification, detection, and segmentation. Many real-life applications, such as self-driving cars, surveillance cameras, and more, use CNNs.

The importance of CNNs

These are several reasons why CNNs are important, as highlighted below:

• Unlike traditional machine learning models like SVM and decision trees that require manual feature extractions, CNNs can perform automatic feature extraction at scale, making them efficient.
• The convolutions layers make CNNs translation invariant, meaning they can recognize patterns from data and extract features regardless of their position, whether the image is rotated, scaled, or shifted.
• Multiple pre-trained CNN models such as VGG-16, ResNet50, Inceptionv3, and EfficientNet are proved to have reached state-of-the-art results and can be fine-tuned on news tasks using a relatively small amount of data.
• CNNs can also be used for non-image classification problems and are not limited to natural language processing, time series analysis, and speech recognition.

Architecture of a CNN

CNNs’ architecture tries to mimic the structure of neurons in the human visual system composed of multiple layers, where each one is responsible for detecting a specific feature in the data. As illustrated in the image below, the typical CNN is made of a combination of four main layers:

• Convolutional layers
• Rectified Linear Unit (ReLU for short)
• Pooling layers
• Fully connected layers

Let’s understand how each of these layers works using the following example of classification of the handwritten digit.

Convolution layers

This is the first building block of a CNN. As the name suggests, the main mathematical task performed is called convolution, which is the application of a sliding window function to a matrix of pixels representing an image. The sliding function applied to the matrix is called kernel or filter, and both can be used interchangeably.

In the convolution layer, several filters of equal size are applied, and each filter is used to recognize a specific pattern from the image, such as the curving of the digits, the edges, the whole shape of the digits, and more.

Let’s consider this 32×32 grayscale image of a handwritten digit. The values in the matrix are given for illustration purposes.

Also, let’s consider the kernel used for the convolution. It is a matrix with a dimension of 3×3. The weights of each element of the kernel is represented in the grid. Zero weights are represented in the black grids and ones in the white grid.

Do we have to manually find these weights?

In real life, the weights of the kernels are determined during the training process of the neural network.

Using these two matrices, we can perform the convolution operation by taking applying the dot product, and work as follows:

1. Apply the kernel matrix from the top-left corner to the right.
2. Perform element-wise multiplication.
3. Sum the values of the products.
4. The resulting value corresponds to the first value (top-left corner) in the convoluted matrix.
5. Move the kernel down with respect to the size of the sliding window.
6. Repeat from step 1 to 5 until the image matrix is fully covered.

The dimension of the convoluted matrix depends on the size of the sliding window. The higher the sliding window, the smaller the dimension.

Another name associated with the kernel in the literature is feature detector because the weights can be fine-tuned to detect specific features in the input image.

For instance:

• Averaging neighboring pixels kernel can be used to blur the input image.
• Subtracting neighboring kernel is used to perform edge detection.

The more convolution layers the network has, the better the layer is at detecting more abstract features.

Activation function

A ReLU activation function is applied after each convolution operation. This function helps the network learn non-linear relationships between the features in the image, hence making the network more robust for identifying different patterns. It also helps to mitigate the vanishing gradient problems.

Pooling layer

The goal of the pooling layer is to pull the most significant features from the convoluted matrix. This is done by applying some aggregation operations, which reduces the dimension of the feature map (convoluted matrix), hence reducing the memory used while training the network. Pooling is also relevant for mitigating overfitting.

The most common aggregation functions that can be applied are:

• Max pooling which is the maximum value of the feature map
• Sum pooling corresponds to the sum of all the values of the feature map
• Average pooling is the average of all the values.

Below is an illustration of each of the previous example:

Also, the dimension of the feature map becomes smaller as the polling function is applied.

The last pooling layer flattens its feature map so that it can be processed by the fully connected layer.

Fully connected layers

These layers are in the last layer of the convolutional neural network, and their inputs correspond to the flattened one-dimensional matrix generated by the last pooling layer. ReLU activations functions are applied to them for non-linearity.

Finally, a softmax prediction layer is used to generate probability values for each of the possible output labels, and the final label predicted is the one with the highest probability score.

Dropout

Dropout is a regularization technic applied to improve the generalization capability of the neural networks with a large number of parameters. It consists of randomly dropping some neurons during the training process, which forces the remaining neurons to learn new features from the input data.

Since the technical implementation will be performed using TensorFlow 2, the next section aims to provide a complete overview of different components of this framework to efficiently build deep learning models.

Setup

Begin by installing and importing some necessary libraries, including:
remotezip to inspect the contents of a ZIP file, tqdm to use a progress bar, OpenCV to process video files, einops for performing more complex tensor operations, and

tensorflow_docs

for embedding data in a Jupyter notebook.


pip install remotezip tqdm opencv-python einops


# Install TensorFlow 2.10


pip install tensorflow==2.10.0



import tqdm import random import pathlib import itertools import collections import cv2 import einops import numpy as np import remotezip as rz import seaborn as sns import matplotlib.pyplot as plt import tensorflow as tf import keras from keras import layers


2023-10-27 01:29:52.291653: W tensorflow/stream_executor/platform/default/dso_loader.cc:64] Could not load dynamic library ‘libcudart.so.11.0’; dlerror: libcudart.so.11.0: cannot open shared object file: No such file or directory; LD_LIBRARY_PATH: /tmpfs/src/tf_docs_env/lib/python3.9/site-packages/cv2/../../lib64: 2023-10-27 01:29:52.327803: E tensorflow/stream_executor/cuda/cuda_blas.cc:2981] Unable to register cuBLAS factory: Attempting to register factory for plugin cuBLAS when one has already been registered 2023-10-27 01:29:52.949253: W tensorflow/stream_executor/platform/default/dso_loader.cc:64] Could not load dynamic library ‘libnvinfer.so.7’; dlerror: libnvinfer.so.7: cannot open shared object file: No such file or directory; LD_LIBRARY_PATH: /tmpfs/src/tf_docs_env/lib/python3.9/site-packages/cv2/../../lib64: 2023-10-27 01:29:52.949379: W tensorflow/stream_executor/platform/default/dso_loader.cc:64] Could not load dynamic library ‘libnvinfer_plugin.so.7’; dlerror: libnvinfer_plugin.so.7: cannot open shared object file: No such file or directory; LD_LIBRARY_PATH: /tmpfs/src/tf_docs_env/lib/python3.9/site-packages/cv2/../../lib64: 2023-10-27 01:29:52.949390: W tensorflow/compiler/tf2tensorrt/utils/py_utils.cc:38] TF-TRT Warning: Cannot dlopen some TensorRT libraries. If you would like to use Nvidia GPU with TensorRT, please make sure the missing libraries mentioned above are installed properly.

Introduction to CNN

Yann LeCun, director of Facebook’s AI Research Group, is the pioneer of convolutional neural networks. He built the first convolutional neural network called LeNet in 1988. LeNet was used for character recognition tasks like reading zip codes and digits.

Have you ever wondered how facial recognition works on social media, or how object detection helps in building self-driving cars, or how disease detection is done using visual imagery in healthcare? It’s all possible thanks to convolutional neural networks (CNN). Here’s an example of convolutional neural networks that illustrates how they work:

Imagine there’s an image of a bird, and you want to identify whether it’s really a bird or some other object. The first thing you do is feed the pixels of the image in the form of arrays to the input layer of the neural network (multi-layer networks used to classify things). The hidden layers carry out feature extraction by performing different calculations and manipulations. There are multiple hidden layers like the convolution layer, the ReLU layer, and pooling layer, that perform feature extraction from the image. Finally, there’s a fully connected layer that identifies the object in the image.

Fig: Convolutional Neural Network to identify the image of a bird

Conclusion

This article has covered a complete overview of CNNs in TensorFlow, providing details about each layer of the CNNs architecture. Also, it made a brief introduction to TensorFlow and how it helps machine learning engineers and researchers build sophisticated neural networks.

We applied all these skill sets to a real-world scenario related to a multiclass classification task.

Our beginner’s guide to object detection could be a great next step to further your learning about computer vision. It explores the key components in object detection and explains how to implement in SSD and Faster RCNN available in Tensorflow.

Python Courses

Course

Intermediate Python

Course

Introduction to Deep Learning in Python

Navigating the World of MLOps Certifications

Multilayer Perceptrons in Machine Learning: A Comprehensive Guide

An End-to-End ML Model Monitoring Workflow with NannyML in Python

Bex Tuychiev

15 min

If you are a software developer who wants to build scalable AI-powered algorithms, you need to understand how to use the tools to build them. This course is part of the upcoming Machine Learning in Tensorflow Specialization and will teach you best practices for using TensorFlow, a popular open-source framework for machine learning.

Convolutional Neural Networks in TensorFlow

This course is part of DeepLearning.AI TensorFlow Developer Professional Certificate

Taught in English

Some content may not be translated

142,698 already enrolled

Convolutional Neural Networks (CNN) in TensorFlow

Now that you understand how convolutional neural networks work, you can start building them using TensorFlow. However, you will first have to install TensorFlow. If you are working on a Google Colab environment, TensorFlow will already be installed.

How to install TensorFlow

TensorFlow can be installed via pip. Run the following command to install it.

<br /> pip install tensorflow<br />

Alternatively, you can run TensorFlow in a container.

<br /> docker pull tensorflow/tensorflow:latest # Download latest stable image docker run -it -p 8888:8888 tensorflow/tensorflow:latest-jupyter # Start Jupyter server<br />

How to confirm TensorFlow is installed

After installation is complete via pip, you might want to check TensorFlow’s version or confirm its installation. If you manage to import TensorFlow without any errors, then it was installed successfully.

<br /> import tensorflow print(tensorflow.__version__)<br />

What are Keras and tf.keras?

As of TensorFlow 2.0, Keras has become the official high-level API for TensorFlow. It is an open-source package that has been integrated into TensorFlow in order to quicken the process of building deep learning models. It is accessible via `tf.keras`. That is what you will be using in this article.

Develop multilayer CNN models

Let’s now take a look at how you can build a convolutional neural network with Keras and TensorFlow. The CIFAR-10 dataset will be used. The dataset contains 60000 32×32 color images in 10 classes, with 6000 images per class.

Develop multilayer CNN models

Loading the dataset can be done directly by using Keras utilities. Other datasets that ship with TensorFlow can be loaded in a similar manner.

<br /> (X_train, y_train), (X_test, y_test) = tf.keras.datasets.cifar10.load_data()<br />

The dataset contains the following classes

<br /> &#8216;airplane&#8217;, &#8216;automobile&#8217;, &#8216;bird&#8217;, &#8216;cat&#8217;, &#8216;deer&#8217;, &#8216;dog&#8217;, &#8216;frog&#8217;, &#8216;horse&#8217;, &#8216;ship&#8217;, &#8216;truck&#8217;<br />

You can use Matplotlib to visualize one of the images. Let’s visualize the image at index 785.

<br /> import matplotlib.pyplot as plt image = X_train[785] plt.imshow(image) plt.show()<br />

That looks like a cat. You can confirm that from the `y_train`. 3 is the label for a cat.

Data preprocessing

The weights of a neural network are initialized to very small numbers. Therefore, scaling the images to be within the same range is important. In this case, let’s scale the values to be numbers between 0 and 1.

<br /> X_train = X_train / 255 X_test = X_test / 255<br />

Build the convolutional neural network

The next step is to define the convolutional neural network. Here is where the convolution, pooling, and flattening layers will be applied. The first layer is the `Conv2D`layer. It’s defined with the following parameters:

• 32 output filters
• a 3 by 3 feature detector
• `same` padding to result in even padding for the input
• input shape of `(32, 32, 3)` because the images are of size 32 by 32. 3 notifies the network that images are colored
• the `relu` activation function so as to achieve non-linearity

The next layer is a max-pooling layer defined with the following parameters:

• a `pool_size` of (2, 2) that defines the size of the pooling window
• 2 strides that define the number of steps taken by the pooling window

Remember that you can design your network as you like. You just have to monitor the metrics and tweak the design and settle on the one that results in the best performance. In this case, another convolution and pooling layer is created. That is followed by the flatten layer whose results are passed to the dense layer. The final layer has 10 units because the dataset has 10 classes. Since it’s a multiclass problem, the Softmax activation function is applied.

<br /> model = tf.keras.Sequential( [ tf.keras.layers.Conv2D(32, (3,3), padding=&#8217;same&#8217;, activation=&#8221;relu&#8221;,input_shape=(32, 32, 3)), tf.keras.layers.MaxPooling2D((2, 2), strides=2), tf.keras.layers.Conv2D(64, (3,3), padding=&#8217;same&#8217;, activation=&#8221;relu&#8221;), tf.keras.layers.MaxPooling2D((2, 2), strides=2), tf.keras.layers.Flatten(), tf.keras.layers.Dense(100, activation=&#8221;relu&#8221;), tf.keras.layers.Dense(10, activation=&#8221;softmax&#8221;) ] )<br />

How to visualize a deep learning model

The quickest way to visualize your model is to use the model summary function.

<br /> model.summary()<br />

You can also use the Keras `plot_model` utility to plot the model.

<br /> tf.keras.utils.plot_model( model, to_file=&#8221;model.png&#8221;, show_shapes=True, show_layer_names=True, rankdir=&#8221;TB&#8221;, expand_nested=True, dpi=96, )<br />

How to reduce overfitting with Dropout

One of the common ways to improve the performance of deep learning models is to introduce dropout regularization. In this process, a specified percentage of connections are dropped during the training process. This forces the network to learn patterns from the data instead of memorizing the data. This is what reduces overfitting. In Keras, this can be achieved by introducing a Dropout layer in the network. Here is how the network would look like after applying the DropOut layer.

<br /> model = tf.keras.Sequential( [ tf.keras.layers.Conv2D(32, (3,3), padding=&#8217;same&#8217;, activation=&#8221;relu&#8221;,input_shape=(32, 32, 3)), tf.keras.layers.MaxPooling2D((2, 2), strides=2), tf.keras.layers.Conv2D(64, (3,3), padding=&#8217;same&#8217;, activation=&#8221;relu&#8221;), tf.keras.layers.MaxPooling2D((2, 2), strides=2), tf.keras.layers.Flatten(), tf.keras.layers.Dense(100, activation=&#8221;relu&#8221;), tf.keras.layers.Dropout(0.2), tf.keras.layers.Dense(10, activation=&#8221;softmax&#8221;) ] )<br />

Compiling the model

The next step is to compile the model. The Sparse Categorical Cross-Entropy loss is used because the labels are not one-hot encoded. In the event that you want to encode the labels, then you will have to use the Categorical Cross-Entropy loss function.

<br /> model.compile(optimizer=&#8217;adam&#8217;, loss=tf.keras.losses.SparseCategoricalCrossentropy(from_logits=True), metrics=[&#8216;accuracy&#8217;])<br />

How to halt training at the right time with Early Stopping

Left to train for more epochs than needed, your model will most likely overfit on the training set. One of the ways to avoid that is to stop the training process when the model stops improving. This is done by monitoring the loss or the accuracy. In order to achieve that, the Keras EarlyStopping callback is used. By default, the callback monitors the validation loss. Patience is the number of epochs to wait before stopping the training process if there is no improvement in the model loss. This callback will be used at the training stage. The callbacks should be passed as a list, even if it’s just one callback.

<br /> from tensorflow.keras.callbacks import EarlyStopping callbacks = [ EarlyStopping(patience=2) ]<br />

How to save the best model automatically

You might also be interested in automatically saving the best model or model weights during training. That can be applied using a Keras ModelCheckpoint callback. The callback will save the best model after each epoch. You can instruct it to save the entire model or just the model weights. By default, it will save the models where the validation loss is minimum.

<br /> checkpoint_filepath = &#8216;/tmp/checkpoint&#8217; model_checkpoint_callback = tf.keras.callbacks.ModelCheckpoint( filepath=checkpoint_filepath, save_weights_only=False, monitor=&#8217;loss&#8217;, mode=&#8217;min&#8217;, save_best_only=True)<br />

Your callbacks will now look like this.

<br /> callbacks = [ EarlyStopping(patience=2), model_checkpoint_callback, ]<br />

After training, you can load the model again using the Keras `load_model` utility.

<br /> another_saved_model = tf.keras.models.load_model(checkpoint_filepath)<br />

Training the model

Let’s now fit the data to the training set. The validation set is passed as well because the callback monitors the validation set. In this case, you can define many epochs but the training process will be stopped by the callback when the loss doesn’t improve after 2 epochs as declared in the EarlyStopping callback.

<br /> history = model.fit(X_train,y_train, epochs=600,validation_data=(X_test,y_test),callbacks=callbacks)<br />

How to plot model learning curves

Learning curves are important because they can inform you whether the model is learning or overfitting. If the validation loss increases significantly or the validation accuracy reduces sharply then your model is most likely overfitting. Since the model was saved into a history variable, you can use that to access the losses and accuracy and plot them. You can also store them in a Pandas DataFrame.

<br /> import pandas as pd metrics_df = pd.DataFrame(history.history)<br />

Let’s now look at how you would plot the training and validation loss.

<br /> metrics_df[[&#8220;loss&#8221;,&#8221;val_loss&#8221;]].plot();<br />

The same can be done for the training and validation accuracy.

<br /> metrics_df[[&#8220;accuracy&#8221;,&#8221;val_accuracy&#8221;]].plot();<br />

How to save and load your model

You might be interested in saving the model for later use. Saving the model is important so that you don’t have to train the model again. This is especially critical for image models that take a long period to train. The H5 format is a common format for saving Keras models.

<br /> model.save(“model.h5”)<br />

The Keras `load_model` is used for loading the model again.

<br /> load_saved_model = tf.keras.models.load_model(&#8220;model.h5&#8221;) load_saved_model.summary()<br />

How to accelerate training with Batch Normalization

The network you trained here was relatively small. However, in other cases, you might have to train a very deep neural network. Training such a network can be very slow. The training process can be hastened using Batch Normalization. It transforms the data ensuring that the mean output is closer to zero and the output standard deviation is close to 1. The mean and variance are computed using the current batch of inputs. Since Batch Normalization offers some form of regularization it is usually not used with DropOut. Here’s how the model would look like after adding the batch normalization layer.

<br /> model = tf.keras.Sequential( [ tf.keras.layers.Conv2D(32, (3,3), padding=&#8217;same&#8217;, activation=&#8221;relu&#8221;,input_shape=(32, 32, 3)), tf.keras.layers.MaxPooling2D((2, 2), strides=2), tf.keras.layers.Conv2D(64, (3,3), padding=&#8217;same&#8217;, activation=&#8221;relu&#8221;), tf.keras.layers.MaxPooling2D((2, 2), strides=2), tf.keras.layers.Flatten(), tf.keras.layers.Dense(100, activation=&#8221;relu&#8221;), tf.keras.layers.BatchNormalization(), tf.keras.layers.Dense(10, activation=&#8221;softmax&#8221;) ]<br />

The working of the batch normalization layer is different during training and during prediction and evaluation. During training `trainable=True` while during prediction and evaluation it’s false. When training normalization is done using the mean and standard deviation of the current batch of inputs. At inference i.e prediction and evaluation, normalization is done using a moving average of the mean and the standard deviation of the batches seen during training. When using a pre-trained model that contains this layer, training for the batch normalization layer has to be set to false. Otherwise the mean and standard deviation will be disrupted and all the prior learning lost.

What are Tensors?

We mainly deal with high-dimensional data when building machine learning and deep learning models. Tensors are multi-dimensional arrays with a uniform type used to represent different features of the data.

Below is the graphical representation of the different types of dimensions of tensors.

• A 0-dimensional tensor contains a single value.
• A 1-dimensional tensor, also known as “rank-1” tensor is list of values.
• A 2-dimensional tensor is a “rank-2” tensor.
• Finally, we can have a N-dimensional tensor, where N represents the number of dimensions within the tensor. In the previous cases, N is respectively 0, 1 and 2.

Below is an illustration of a zero to a 3-dimensional tensor. Each tensor is created using the constant() function from TensorFlow.


# Zero dimensional tensor zero_dim_tensor = tf.constant(20) print(zero_dim_tensor) # One dimensional tensor one_dim_tensor = tf.constant([12, 20, 53, 26, 11, 56]) print(one_dim_tensor) # Two dimensional tensor two_dim_array = [[3, 6, 7, 5], [9, 2, 3, 4], [7, 1, 10,6], [0, 8, 11,2]] two_dim_tensor = tf.constant(two_dim_array) print(two_dim_tensor)


A successful execution of the previous code should generate the outputs below, and we can notice the keyword “tf.Tensor” to mean that the result is a tensor. It has three parameters:

• The actual value of the tensor.
• The shape() of the tensor, which is 0, 6 by 1, and 4 by 4, respectively for the first, second, and third tensors.
• The data type represented by the dtype attribute, and all the tensors are int32.

Our Tensorflow Tutorial for Beginners provides a complete overview of TensorFlow and teaches how to build and train models.

Architectures of CNNs

You don’t always have to design your convolutional neural networks from scratch. Other times one can try architectures developed by experts. These have proven to perform well on many image tasks. Some of these architectures are:

They can be accessed via Keras applications. These applications have also been pre-trained on the ImageNet dataset. The dataset contains over a million images. This makes these applications robust enough for use in the real world. When instantiating the model, you have the choice whether to include the pre-trained weights or not. When the weights are used, you can start using the model for classification right away. Other ways of using the pre-trained models are:

• extracting features and passing them to a new model
• fine-tuning a new model

Let’s take a look at how you can load the Xception architecture without weights. Since weights are not included, you can use your dataset to train the model.

<br /> model = tf.keras.applications.Xception( include_top=True, input_tensor=None, input_shape=None, pooling=None, classes=1000, classifier_activation=&#8221;softmax&#8221;, )<br />

When you load the model with weights, you can start using it for prediction right away. The weights are stored in this location `~/.keras/models/`.

<br /> model = tf.keras.applications.Xception( include_top=True, weights=&#8221;imagenet&#8221;, input_tensor=None, input_shape=None, pooling=None, classes=1000, classifier_activation=&#8221;softmax&#8221;, )<br />

After that, you can process the image and run the predictions. The Keras applications provide a function for doing that. Each of the architectures dictates the size of the image that should be passed to it. You should always confirm that from its documentation. Next, convert the image into an array and expand its dimensions in order to include the batch size.

<br /> from tensorflow.keras.preprocessing import image import numpy as np !wget &#8211;no-check-certificate \ https://upload.wikimedia.org/wikipedia/commons/b/b5/Lion_d%27Afrique.jpg \ -O /tmp/lion.jpg img_path = &#8216;/tmp/lion.jpg&#8217; img = image.load_img(img_path, target_size=(299, 299)) x = image.img_to_array(img) x = np.expand_dims(x, axis=0) x = tf.keras.applications.xception.preprocess_input(x) preds = model.predict(x) # decode the results into a list of tuples (class, description, probability) # (one such list for each sample in the batch) print(&#8216;Predicted:&#8217;, tf.keras.applications.xception.decode_predictions(preds, top=3)[0])<br />

The final step is to decode the predictions and print the results.

TensorFlow: Constants, Variables, and Placeholders

Constants are not the only types of tensors. There are also variables and placeholders, which are all building blocks of a computational graph.

A computational graph is basically and a representation of a sequence of operations and the flow of data between them.

Now, let’s understand the difference between these types of tensors.

Constants

Constants are tensors whose values do not change during the execution of the computational graph. They are created using the tf.constant() function and are mainly used to store fixed parameters that do not require any change during the model training.

Variables

Variables are tensors whose value can be changed during the execution of the computational graph and they are created using the tf.Variable() function. For instance, in the case of neural networks, weights, and biases can be defined as variables since they need to be updated during the training process.

Placeholders

These were used in the first version of Tensorflow as empty containers that do not have specific values. They are just used to reverse a spot for data to be used in the future. This gives the users the freedom to use different datasets and batch sizes during model training and validation.

In Tensorflow version 2, placeholders have been replaced by the tf.function() function, which is a more Pythonic and dynamic approach to feeding data into the computational graph.

Making predictions

You can now use this image to run a prediction.

<br /> prediction = model.predict(test_image)<br />

When you print this you will see something similar to this.

<br /> prediction[0][0] 0.014393696<br />

The question is, how do you interpret this? Remember that the network output layer has just one unit and uses the sigmoid activation function. The output of this network is therefore a number between 0 and 1. That number represents the probability that the image belongs to class 1. Class 1 in this case is dogs. You can therefore set a threshold of say 50% to separate the two classes.

<br /> if prediction[0][0]&gt;0.5: print(&#8221; is a dog&#8221;) else: print(&#8221; is a cat&#8221;)<br />

Since the obtained probability is less than 0.5 then that image is definitely that of a cat.

You can repeat the same process with a dogs image. First, start by downloading the image.

<br /> !wget &#8211;no-check-certificate \ https://upload.wikimedia.org/wikipedia/commons/1/18/Dog_Breeds.jpg \ -O /tmp/dog.jpg<br />

After that, load it while converting it to the required size.

<br /> test_image2 = image.load_img(&#8216;/tmp/dog.jpg&#8217;, target_size=(200, 200))<br />

Next, expand the dimensions and run the prediction.

<br /> test_image2 = np.expand_dims(test_image2, axis=0) prediction = model.predict(test_image2)<br />

Use the same threshold to determine if it is the image of a cat or a dog.

<br /> if prediction[0][0]&gt;0.5: print(&#8221; is a dog&#8221;) else: print(&#8221; is a cat&#8221;)<br />

With an accuracy of 99%, the image is classified as a dog.

Use case implementation using CNN

We’ll be using the CIFAR-10 dataset from the Canadian Institute For Advanced Research for classifying images across 10 categories using CNN.

1. Download the data set:

2. Import the CIFAR data set:

3. Read the label names:

4. Display the images using matplotlib:

5. Use the helper function to handle data:

6. Create the model:

7. Apply the helper functions:

8. Create the layers for convolution and pooling:

9. Create the flattened layer by reshaping the pooling layer:

10. Create a fully connected layer:

11. Set the output to y_pred variable:

12. Apply the loss function:

13. Create the optimizer:

14. Create a variable to initialize all the global variables:

15. Run the model by creating a graph session:

Build deep learning models in TensorFlow and learn the TensorFlow open-source framework with the Deep Learning Course (with Keras &TensorFlow). Enroll now!

Final remarks

In this article, you have learned CNNs from their intuition to their applications in the real world. You have also seen that you can use existing architectures to hasten your model development process. Specifically, you have covered:

• what convolutional neural networks are
• how convolutional neural networks work
• using pre-trained convolutional neural networks to run image classification
• building convolutional neural networks from scratch using Keras and TensorFlow
• how to plot the learning curves of your neural network
• preventing overfitting using DropOut regularization and batch normalization
• saving your best model using the model checkpoint callback
• how to stop the training process of your CNN when it stops improving
• how you can save and load the model again

…just to mention a few.

And that’s not the end of it, you can explore all the examples used in this article on this Google Colab Notebook. Feel free to play with the parameters of the models to see how they affect the performance of the model.

• TensorFlow Tutorial
• TensorFlow – Home
• TensorFlow – Introduction
• TensorFlow – Installation
• Understanding Artificial Intelligence
• Mathematical Foundations
• Machine Learning & Deep Learning
• TensorFlow – Basics
• Convolutional Neural Networks
• Recurrent Neural Networks
• TensorBoard Visualization
• TensorFlow – Word Embedding
• Single Layer Perceptron
• TensorFlow – Linear Regression
• TFLearn and its installation
• CNN and RNN Difference
• TensorFlow – Keras
• TensorFlow – Distributed Computing
• TensorFlow – Exporting
• Multi-Layer Perceptron Learning
• Hidden Layers of Perceptron
• TensorFlow – Optimizers
• TensorFlow – XOR Implementation
• Gradient Descent Optimization
• TensorFlow – Forming Graphs
• Image Recognition using TensorFlow
• Recommendations for Neural Network Training

• TensorFlow Useful Resources
• TensorFlow – Quick Guide
• TensorFlow – Useful Resources
• TensorFlow – Discussion

TensorFlow – Convolutional Neural Networks

After understanding machine-learning concepts, we can now shift our focus to deep learning concepts. Deep learning is a division of machine learning and is considered as a crucial step taken by researchers in recent decades. The examples of deep learning implementation include applications like image recognition and speech recognition.

Following are the two important types of deep neural networks −

• Convolutional Neural Networks
• Recurrent Neural Networks

In this chapter, we will focus on the CNN, Convolutional Neural Networks.

CNN on TensorFlow Concepts

Tensor

Tensors represent deep learning data. They are multidimensional arrays, used to store multiple dimensions of a dataset. Each dimension is called a feature. For example, a cube storing data across an X, Y, and Z access is represented as a 3-dimensional tensor. Tensors can store very high dimensionality, with hundreds of dimensions of features typically used in deep learning applications.

Computational graph

TensorFlow computational graphs represent the workflows that occur during deep learning model training. For a CNN model, the computational graph can be very complex. The image below demonstrates how a simple graph should look like. You can use TensorBoard, built into TensorFlow, to display the computational graph of your model.

Constant

In TensorFlow, a constant is used to store values that don’t change during the computation of the model. It is used for nodes that must remain the same during model training. A constant does not have parameters.

Placeholder

Placeholders are used to input training examples to your deep learning model. A placeholder can take parameters, and these parameters are changed at runtime as the model processes the training set.

Variable

Variables are used to add trainable nodes to the computation graph, such as weights and biases.

Related content: read our guide to deep convolutional neural networks.

TensorFlow Implementation of CNN

In this section, we will learn about the TensorFlow implementation of CNN. The steps,which require the execution and proper dimension of the entire network, are as shown below −

Step 1 − Include the necessary modules for TensorFlow and the data set modules, which are needed to compute the CNN model.

import tensorflow as tf import numpy as np from tensorflow.examples.tutorials.mnist import input_data

Step 2 − Declare a function called run_cnn(), which includes various parameters and optimization variables with declaration of data placeholders. These optimization variables will declare the training pattern.

def run_cnn(): mnist = input_data.read_data_sets(“MNIST_data/”, one_hot = True) learning_rate = 0.0001 epochs = 10 batch_size = 50

Step 3 − In this step, we will declare the training data placeholders with input parameters – for 28 x 28 pixels = 784. This is the flattened image data that is drawn from mnist.train.nextbatch().

We can reshape the tensor according to our requirements. The first value (-1) tells function to dynamically shape that dimension based on the amount of data passed to it. The two middle dimensions are set to the image size (i.e. 28 x 28).

x = tf.placeholder(tf.float32, [None, 784]) x_shaped = tf.reshape(x, [-1, 28, 28, 1]) y = tf.placeholder(tf.float32, [None, 10])

Step 4 − Now it is important to create some convolutional layers −

layer1 = create_new_conv_layer(x_shaped, 1, 32, [5, 5], [2, 2], name = ‘layer1’) layer2 = create_new_conv_layer(layer1, 32, 64, [5, 5], [2, 2], name = ‘layer2’)

Step 5 − Let us flatten the output ready for the fully connected output stage – after two layers of stride 2 pooling with the dimensions of 28 x 28, to dimension of 14 x 14 or minimum 7 x 7 x,y co-ordinates, but with 64 output channels. To create the fully connected with “dense” layer, the new shape needs to be [-1, 7 x 7 x 64]. We can set up some weights and bias values for this layer, then activate with ReLU.

flattened = tf.reshape(layer2, [-1, 7 * 7 * 64]) wd1 = tf.Variable(tf.truncated_normal([7 * 7 * 64, 1000], stddev = 0.03), name = ‘wd1’) bd1 = tf.Variable(tf.truncated_normal([1000], stddev = 0.01), name = ‘bd1’) dense_layer1 = tf.matmul(flattened, wd1) + bd1 dense_layer1 = tf.nn.relu(dense_layer1)

Step 6 − Another layer with specific softmax activations with the required optimizer defines the accuracy assessment, which makes the setup of initialization operator.

wd2 = tf.Variable(tf.truncated_normal([1000, 10], stddev = 0.03), name = ‘wd2’) bd2 = tf.Variable(tf.truncated_normal([10], stddev = 0.01), name = ‘bd2’) dense_layer2 = tf.matmul(dense_layer1, wd2) + bd2 y_ = tf.nn.softmax(dense_layer2) cross_entropy = tf.reduce_mean( tf.nn.softmax_cross_entropy_with_logits(logits = dense_layer2, labels = y)) optimiser = tf.train.AdamOptimizer(learning_rate = learning_rate).minimize(cross_entropy) correct_prediction = tf.equal(tf.argmax(y, 1), tf.argmax(y_, 1)) accuracy = tf.reduce_mean(tf.cast(correct_prediction, tf.float32)) init_op = tf.global_variables_initializer()

Step 7 − We should set up recording variables. This adds up a summary to store the accuracy of data.

tf.summary.scalar(‘accuracy’, accuracy) merged = tf.summary.merge_all() writer = tf.summary.FileWriter(‘E:\TensorFlowProject’) with tf.Session() as sess: sess.run(init_op) total_batch = int(len(mnist.train.labels) / batch_size) for epoch in range(epochs): avg_cost = 0 for i in range(total_batch): batch_x, batch_y = mnist.train.next_batch(batch_size = batch_size) _, c = sess.run([optimiser, cross_entropy], feed_dict = { x:batch_x, y: batch_y}) avg_cost += c / total_batch test_acc = sess.run(accuracy, feed_dict = {x: mnist.test.images, y: mnist.test.labels}) summary = sess.run(merged, feed_dict = {x: mnist.test.images, y: mnist.test.labels}) writer.add_summary(summary, epoch) print(“\nTraining complete!”) writer.add_graph(sess.graph) print(sess.run(accuracy, feed_dict = {x: mnist.test.images, y: mnist.test.labels})) def create_new_conv_layer( input_data, num_input_channels, num_filters,filter_shape, pool_shape, name): conv_filt_shape = [ filter_shape[0], filter_shape[1], num_input_channels, num_filters] weights = tf.Variable( tf.truncated_normal(conv_filt_shape, stddev = 0.03), name = name+’_W’) bias = tf.Variable(tf.truncated_normal([num_filters]), name = name+’_b’) #Out layer defines the output out_layer = tf.nn.conv2d(input_data, weights, [1, 1, 1, 1], padding = ‘SAME’) out_layer += bias out_layer = tf.nn.relu(out_layer) ksize = [1, pool_shape[0], pool_shape[1], 1] strides = [1, 2, 2, 1] out_layer = tf.nn.max_pool( out_layer, ksize = ksize, strides = strides, padding = ‘SAME’) return out_layer if __name__ == “__main__”: run_cnn()

Following is the output generated by the above code −

See @{tf.nn.softmax_cross_entropy_with_logits_v2}. 2018-09-19 17:22:58.802268: I T:\src\github\tensorflow\tensorflow\core\platform\cpu_feature_guard.cc:140] Your CPU supports instructions that this TensorFlow binary was not compiled to use: AVX2 2018-09-19 17:25:41.522845: W T:\src\github\tensorflow\tensorflow\core\framework\allocator.cc:101] Allocation of 1003520000 exceeds 10% of system memory. 2018-09-19 17:25:44.630941: W T:\src\github\tensorflow\tensorflow\core\framework\allocator.cc:101] Allocation of 501760000 exceeds 10% of system memory. Epoch: 1 cost = 0.676 test accuracy: 0.940 2018-09-19 17:26:51.987554: W T:\src\github\tensorflow\tensorflow\core\framework\allocator.cc:101] Allocation of 1003520000 exceeds 10% of system memory.

What are Tensors?

We mainly deal with high-dimensional data when building machine learning and deep learning models. Tensors are multi-dimensional arrays with a uniform type used to represent different features of the data.

Below is the graphical representation of the different types of dimensions of tensors.

• A 0-dimensional tensor contains a single value.
• A 1-dimensional tensor, also known as “rank-1” tensor is list of values.
• A 2-dimensional tensor is a “rank-2” tensor.
• Finally, we can have a N-dimensional tensor, where N represents the number of dimensions within the tensor. In the previous cases, N is respectively 0, 1 and 2.

Below is an illustration of a zero to a 3-dimensional tensor. Each tensor is created using the constant() function from TensorFlow.


# Zero dimensional tensor zero_dim_tensor = tf.constant(20) print(zero_dim_tensor) # One dimensional tensor one_dim_tensor = tf.constant([12, 20, 53, 26, 11, 56]) print(one_dim_tensor) # Two dimensional tensor two_dim_array = [[3, 6, 7, 5], [9, 2, 3, 4], [7, 1, 10,6], [0, 8, 11,2]] two_dim_tensor = tf.constant(two_dim_array) print(two_dim_tensor)


A successful execution of the previous code should generate the outputs below, and we can notice the keyword “tf.Tensor” to mean that the result is a tensor. It has three parameters:

• The actual value of the tensor.
• The shape() of the tensor, which is 0, 6 by 1, and 4 by 4, respectively for the first, second, and third tensors.
• The data type represented by the dtype attribute, and all the tensors are int32.

Our Tensorflow Tutorial for Beginners provides a complete overview of TensorFlow and teaches how to build and train models.

CNN Step-by-Step Implementation

Let’s put everything we have learned previously into practice. This section will illustrate the end-to-end implementation of a convolutional neural network in TensorFlow applied to the CIFAR-10 dataset, which is a built-in dataset with the following properties:

• It contains 60.000 32 by 32 color images
• The dataset has 10 different classes
• Each class has 6000 images
• There are overall 50.000 training images
• And overall 10.000 testing images

The source code of the article is available on DataCamp’s workspace

Architecture of the network

Before getting into the technical implementation, let’s first understand the overall architecture of the network being implemented.

• The input of the model is a 32x32x3 tensor, respectively, for the width, height, and channels.
• We will have two convolutional layers. The first layer applies 32 filters of size 3×3 each and a ReLU activation function. And the second one applies 64 filters of size 3×3
• The first pooling layer will apply a 2×2 max pooling
• The second pooling layer will apply a 2×2 max pooling as well
• The fully connected layer will have 128 units and a ReLU activation function
• Finally, the output will be 10 units corresponding to the 10 classes, and the activation function is a softmax to generate the probability distributions.

Load dataset

The built-in dataset is loaded from the keras.datasets() as follows:


(train_images, train_labels), (test_images, test_labels) = cf10.load_data()


Exploratory Data Analysis

In this section, we will focus solely on showing some sample images since we already know the proportion of each class in both the training and testing data.

The helper function show_images() shows a total of 12 images by default and takes three main parameters:

• The training images
• The class names
• And the training labels.


import matplotlib.pyplot as plt def show_images(train_images, class_names, train_labels, nb_samples = 12, nb_row = 4): plt.figure(figsize=(12, 12)) for i in range(nb_samples): plt.subplot(nb_row, nb_row, i + 1) plt.xticks([]) plt.yticks([]) plt.grid(False) plt.imshow(train_images[i], cmap=plt.cm.binary) plt.xlabel(class_names[train_labels[i][0]]) plt.show()


Now, we can call the function with the required parameters.


class_names = ['airplane', 'automobile', 'bird', 'cat', 'deer', 'dog', 'frog', 'horse', 'ship', 'truck'] show_images(train_images, class_names, train_labels)


A successful execution of the previous code generates the images below.

Data preprocessing

Prior to training the model, we need to normalize the pixel values of the data in the same range (e.g. 0 to 1). This is a common preprocessing step when dealing with images to ensure scale invariance, and faster convergence during the training.


max_pixel_value = 255 train_images = train_images / max_pixel_value test_images = test_images / max_pixel_value


Also, we notice that the labels are represented in a categorical format like cat, horse, bird, and so one. We need to convert them into a numerical format so that they can be easily processed by the neural network.


from tensorflow.keras.utils import to_categorical train_labels = to_categorical(train_labels, len(class_names)) test_labels = to_categorical(test_labels, len(class_names))


Model architecture implementation

The next step is to implement the architecture of the network based on the previous description.

First, we define the model using the Sequential() class, and each layer is added to the model with the add() function.


from tensorflow.keras import Sequential from tensorflow.keras.layers import Conv2D, MaxPooling2D, Flatten, Dense # Variables INPUT_SHAPE = (32, 32, 3) FILTER1_SIZE = 32 FILTER2_SIZE = 64 FILTER_SHAPE = (3, 3) POOL_SHAPE = (2, 2) FULLY_CONNECT_NUM = 128 NUM_CLASSES = len(class_names) # Model architecture implementation model = Sequential() model.add(Conv2D(FILTER1_SIZE, FILTER_SHAPE, activation='relu', input_shape=INPUT_SHAPE)) model.add(MaxPooling2D(POOL_SHAPE)) model.add(Conv2D(FILTER2_SIZE, FILTER_SHAPE, activation='relu')) model.add(MaxPooling2D(POOL_SHAPE)) model.add(Flatten()) model.add(Dense(FULLY_CONNECT_NUM, activation='relu')) model.add(Dense(NUM_CLASSES, activation='softmax'))


After applying the summary() function to the model, we a comprehensive summary of the model’s architecture with information about each layer, its type, output shape and the total number of trainable parameters.

Model training

All the resources are finally available to configure and trigger the training of the model. This is done respectively with the compile() and fit() functions which takes the following parameters:

• The Optimizer is responsible for updating the model’s weights and biases. In our case, we are using the Adam optimizer.
• The loss function is used to measure the misclassification errors, and we are using the Crosentropy().
• Finally, the metrics is used to measure the performance of the model, and accuracy, precision, and recall will be displayed in our use case.


from tensorflow.keras.metrics import Precision, Recall BATCH_SIZE = 32 EPOCHS = 30 METRICS = metrics=['accuracy', Precision(name='precision'), Recall(name='recall')] model.compile(optimizer='adam', loss='categorical_crossentropy', metrics = METRICS) # Train the model training_history = model.fit(train_images, train_labels, epochs=EPOCHS, batch_size=BATCH_SIZE, validation_data=(test_images, test_labels))


Model evaluation

After the model training, we can compare its performance on both the training and testing datasets by plotting the above metrics using the show_performance_curve() helper function in two dimensions.

• The horizontal axis (x) is the number of epochs
• The vertical one (y) is the underlying performance of the model.
• The curve represents the value of the metrics at a specific epoch.

For better visualization, a vertical red line is drawn through the intersection of the training and validation performance values along with the optimal value.


def show_performance_curve(training_result, metric, metric_label): train_perf = training_result.history[str(metric)] validation_perf = training_result.history['val_'+str(metric)] intersection_idx = np.argwhere(np.isclose(train_perf, validation_perf, atol=1e-2)).flatten()[0] intersection_value = train_perf[intersection_idx] plt.plot(train_perf, label=metric_label) plt.plot(validation_perf, label = 'val_'+str(metric)) plt.axvline(x=intersection_idx, color='r', linestyle='--', label='Intersection') plt.annotate(f'Optimal Value: {intersection_value:.4f}', xy=(intersection_idx, intersection_value), xycoords='data', fontsize=10, color='green') plt.xlabel('Epoch') plt.ylabel(metric_label) plt.legend(loc='lower right')


Then, the function is applied for both the accuracy and the precision of the model.


show_performance_curve(training_history, 'accuracy', 'accuracy')



show_performance_curve(training_history, 'precision', 'precision')


After training the model without any fine-tuning and pre-processing, we end up with:

• An accuracy score of 67.09%, meaning that the model correctly classifies 67% of the samples out of every 100 samples.
• And, a precision of 76.55%, meaning that out of each 100 positive predictions, almost 77 of them are true positives, and the remaining 23 are false positives.
• These scores are achieved respectively at the third and second epochs for accuracy and precision.

These two metrics give a global understanding of the model behavior.

What if we want to know for each class which ones are the model good at predicting and those that the model struggles with?

This can be achieved from the confusion matrix, which shows for each class the number of correct and wrong predictions. The implementation is given below. We start by making predictions on the test data, then compute the confusion matrix and show the final result.


from sklearn.metrics import confusion_matrix, ConfusionMatrixDisplay test_predictions = model.predict(test_images) test_predicted_labels = np.argmax(test_predictions, axis=1) test_true_labels = np.argmax(test_labels, axis=1) cm = confusion_matrix(test_true_labels, test_predicted_labels) cmd = ConfusionMatrixDisplay(confusion_matrix=cm) cmd.plot(include_values=True, cmap='viridis', ax=None, xticks_rotation='horizontal') plt.show()


• Classes 0, 1, 6, 7, 8, 9, respectively, for airplane, automobile, frog, horse, ship, and truck have the highest values at the diagonal. This means that the model is better at predicting those classes.
• On the other hand, it seems to struggle with the remaining classes:
• The classes with the highest off-diagonal values are those the model confuses the good classes with. For instance, it confuses birds (class 2) with an airplane, and automobile with trucks (class 9).

Learn more about confusion matrix from our tutorial understanding confusion matrix in R, which takes course material from DataCamp’s Machine Learning toolbox course.

This model can be improved with additional tasks such as:

• Image augmentation
• Transfer learning using pre-trained models such as ResNet, MobileNet, or VGG. Our Transfer learning tutorial explains what transfer learning is and some of its applications in real life.
• Applying different regularization technics such as L1, L2 or dropout.
• Fine-tuning different hyperparameters such as learning rate, the batch size, number of layers in the network.

Load and preprocess video data

The hidden cell below defines helper functions to download a slice of data from the UCF-101 dataset, and load it into a

tf.data.Dataset

. You can learn more about the specific preprocessing steps in the Loading video data tutorial, which walks you through this code in more detail.

The

FrameGenerator

class at the end of the hidden block is the most important utility here. It creates an iterable object that can feed data into the TensorFlow data pipeline. Specifically, this class contains a Python generator that loads the video frames along with its encoded label. The generator (

__call__

) function yields the frame array produced by

frames_from_video_file

and a one-hot encoded vector of the label associated with the set of frames.


def list_files_per_class(zip_url): """ List the files in each class of the dataset given the zip URL. Args: zip_url: URL from which the files can be unzipped. Return: files: List of files in each of the classes. """ files = [] with rz.RemoteZip(URL) as zip: for zip_info in zip.infolist(): files.append(zip_info.filename) return files def get_class(fname): """ Retrieve the name of the class given a filename. Args: fname: Name of the file in the UCF101 dataset. Return: Class that the file belongs to. """ return fname.split('_')[-3] def get_files_per_class(files): """ Retrieve the files that belong to each class. Args: files: List of files in the dataset. Return: Dictionary of class names (key) and files (values). """ files_for_class = collections.defaultdict(list) for fname in files: class_name = get_class(fname) files_for_class[class_name].append(fname) return files_for_class def download_from_zip(zip_url, to_dir, file_names): """ Download the contents of the zip file from the zip URL. Args: zip_url: Zip URL containing data. to_dir: Directory to download data to. file_names: Names of files to download. """ with rz.RemoteZip(zip_url) as zip: for fn in tqdm.tqdm(file_names): class_name = get_class(fn) zip.extract(fn, str(to_dir / class_name)) unzipped_file = to_dir / class_name / fn fn = pathlib.Path(fn).parts[-1] output_file = to_dir / class_name / fn unzipped_file.rename(output_file,) def split_class_lists(files_for_class, count): """ Returns the list of files belonging to a subset of data as well as the remainder of files that need to be downloaded. Args: files_for_class: Files belonging to a particular class of data. count: Number of files to download. Return: split_files: Files belonging to the subset of data. remainder: Dictionary of the remainder of files that need to be downloaded. """ split_files = [] remainder = {} for cls in files_for_class: split_files.extend(files_for_class[cls][:count]) remainder[cls] = files_for_class[cls][count:] return split_files, remainder def download_ufc_101_subset(zip_url, num_classes, splits, download_dir): """ Download a subset of the UFC101 dataset and split them into various parts, such as training, validation, and test. Args: zip_url: Zip URL containing data. num_classes: Number of labels. splits: Dictionary specifying the training, validation, test, etc. (key) division of data (value is number of files per split). download_dir: Directory to download data to. Return: dir: Posix path of the resulting directories containing the splits of data. """ files = list_files_per_class(zip_url) for f in files: tokens = f.split('/') if len(tokens) <= 2: files.remove(f) # Remove that item from the list if it does not have a filename files_for_class = get_files_per_class(files) classes = list(files_for_class.keys())[:num_classes] for cls in classes: new_files_for_class = files_for_class[cls] random.shuffle(new_files_for_class) files_for_class[cls] = new_files_for_class # Only use the number of classes you want in the dictionary files_for_class = {x: files_for_class[x] for x in list(files_for_class)[:num_classes]} dirs = {} for split_name, split_count in splits.items(): print(split_name, ":") split_dir = download_dir / split_name split_files, files_for_class = split_class_lists(files_for_class, split_count) download_from_zip(zip_url, split_dir, split_files) dirs[split_name] = split_dir return dirs def format_frames(frame, output_size): """ Pad and resize an image from a video. Args: frame: Image that needs to resized and padded. output_size: Pixel size of the output frame image. Return: Formatted frame with padding of specified output size. """ frame = tf.image.convert_image_dtype(frame, tf.float32) frame = tf.image.resize_with_pad(frame, *output_size) return frame def frames_from_video_file(video_path, n_frames, output_size = (224,224), frame_step = 15): """ Creates frames from each video file present for each category. Args: video_path: File path to the video. n_frames: Number of frames to be created per video file. output_size: Pixel size of the output frame image. Return: An NumPy array of frames in the shape of (n_frames, height, width, channels). """ # Read each video frame by frame result = [] src = cv2.VideoCapture(str(video_path)) video_length = src.get(cv2.CAP_PROP_FRAME_COUNT) need_length = 1 + (n_frames - 1) * frame_step if need_length > video_length: start = 0 else: max_start = video_length - need_length start = random.randint(0, max_start + 1) src.set(cv2.CAP_PROP_POS_FRAMES, start) # ret is a boolean indicating whether read was successful, frame is the image itself ret, frame = src.read() result.append(format_frames(frame, output_size)) for _ in range(n_frames - 1): for _ in range(frame_step): ret, frame = src.read() if ret: frame = format_frames(frame, output_size) result.append(frame) else: result.append(np.zeros_like(result[0])) src.release() result = np.array(result)[..., [2, 1, 0]] return result class FrameGenerator: def __init__(self, path, n_frames, training = False): """ Returns a set of frames with their associated label. Args: path: Video file paths. n_frames: Number of frames. training: Boolean to determine if training dataset is being created. """ self.path = path self.n_frames = n_frames self.training = training self.class_names = sorted(set(p.name for p in self.path.iterdir() if p.is_dir())) self.class_ids_for_name = dict((name, idx) for idx, name in enumerate(self.class_names)) def get_files_and_class_names(self): video_paths = list(self.path.glob('*/*.avi')) classes = [p.parent.name for p in video_paths] return video_paths, classes def __call__(self): video_paths, classes = self.get_files_and_class_names() pairs = list(zip(video_paths, classes)) if self.training: random.shuffle(pairs) for path, name in pairs: video_frames = frames_from_video_file(path, self.n_frames) label = self.class_ids_for_name[name] # Encode labels yield video_frames, label



URL = 'https://storage.googleapis.com/thumos14_files/UCF101_videos.zip' download_dir = pathlib.Path('./UCF101_subset/') subset_paths = download_ufc_101_subset(URL, num_classes = 10, splits = {"train": 30, "val": 10, "test": 10}, download_dir = download_dir)


train : 100%|██████████| 300/300 [00:28<00:00, 10.61it/s] val : 100%|██████████| 100/100 [00:09<00:00, 10.75it/s] test : 100%|██████████| 100/100 [00:08<00:00, 12.31it/s]

Create the training, validation, and test sets (

train_ds

,

val_ds

, and

test_ds

).


n_frames = 10 batch_size = 8 output_signature = (tf.TensorSpec(shape = (None, None, None, 3), dtype = tf.float32), tf.TensorSpec(shape = (), dtype = tf.int16)) train_ds = tf.data.Dataset.from_generator(FrameGenerator(subset_paths['train'], n_frames, training=True), output_signature = output_signature) # Batch the data train_ds = train_ds.batch(batch_size) val_ds = tf.data.Dataset.from_generator(FrameGenerator(subset_paths['val'], n_frames), output_signature = output_signature) val_ds = val_ds.batch(batch_size) test_ds = tf.data.Dataset.from_generator(FrameGenerator(subset_paths['test'], n_frames), output_signature = output_signature) test_ds = test_ds.batch(batch_size)


2023-10-27 01:30:41.429812: W tensorflow/stream_executor/platform/default/dso_loader.cc:64] Could not load dynamic library ‘libcudart.so.11.0’; dlerror: libcudart.so.11.0: cannot open shared object file: No such file or directory; LD_LIBRARY_PATH: /tmpfs/src/tf_docs_env/lib/python3.9/site-packages/cv2/../../lib64: 2023-10-27 01:30:41.429949: W tensorflow/stream_executor/platform/default/dso_loader.cc:64] Could not load dynamic library ‘libcublas.so.11’; dlerror: libcublas.so.11: cannot open shared object file: No such file or directory; LD_LIBRARY_PATH: /tmpfs/src/tf_docs_env/lib/python3.9/site-packages/cv2/../../lib64: 2023-10-27 01:30:41.430034: W tensorflow/stream_executor/platform/default/dso_loader.cc:64] Could not load dynamic library ‘libcublasLt.so.11’; dlerror: libcublasLt.so.11: cannot open shared object file: No such file or directory; LD_LIBRARY_PATH: /tmpfs/src/tf_docs_env/lib/python3.9/site-packages/cv2/../../lib64: 2023-10-27 01:30:41.430115: W tensorflow/stream_executor/platform/default/dso_loader.cc:64] Could not load dynamic library ‘libcufft.so.10’; dlerror: libcufft.so.10: cannot open shared object file: No such file or directory; LD_LIBRARY_PATH: /tmpfs/src/tf_docs_env/lib/python3.9/site-packages/cv2/../../lib64: 2023-10-27 01:30:41.496040: W tensorflow/stream_executor/platform/default/dso_loader.cc:64] Could not load dynamic library ‘libcusparse.so.11’; dlerror: libcusparse.so.11: cannot open shared object file: No such file or directory; LD_LIBRARY_PATH: /tmpfs/src/tf_docs_env/lib/python3.9/site-packages/cv2/../../lib64: 2023-10-27 01:30:41.496279: W tensorflow/core/common_runtime/gpu/gpu_device.cc:1934] Cannot dlopen some GPU libraries. Please make sure the missing libraries mentioned above are installed properly if you would like to use GPU. Follow the guide at https://www.tensorflow.org/install/gpu for how to download and setup the required libraries for your platform. Skipping registering GPU devices…

Convolutional Neural Networks

Convolutional Neural networks are designed to process data through multiple layers of arrays. This type of neural networks is used in applications like image recognition or face recognition. The primary difference between CNN and any other ordinary neural network is that CNN takes input as a two-dimensional array and operates directly on the images rather than focusing on feature extraction which other neural networks focus on.

The dominant approach of CNN includes solutions for problems of recognition. Top companies like Google and Facebook have invested in research and development towards recognition projects to get activities done with greater speed.

A convolutional neural network uses three basic ideas −

• Local respective fields
• Convolution
• Pooling

Let us understand these ideas in detail.

CNN utilizes spatial correlations that exist within the input data. Each concurrent layer of a neural network connects some input neurons. This specific region is called local receptive field. Local receptive field focusses on the hidden neurons. The hidden neurons process the input data inside the mentioned field not realizing the changes outside the specific boundary.

Following is a diagram representation of generating local respective fields −

If we observe the above representation, each connection learns a weight of the hidden neuron with an associated connection with movement from one layer to another. Here, individual neurons perform a shift from time to time. This process is called “convolution”.

The mapping of connections from the input layer to the hidden feature map is defined as “shared weights” and bias included is called “shared bias”.

CNN or convolutional neural networks use pooling layers, which are the layers, positioned immediately after CNN declaration. It takes the input from the user as a feature map that comes out of convolutional networks and prepares a condensed feature map. Pooling layers helps in creating layers with neurons of previous layers.

Tensors vs Matrices: Differences

Many people confuse tensors with matrices. Even though these two objects look similar, they have completely different properties. This section provides a better understanding of the difference between matrices and tensors.

• We can think of a matrice as a tensor with only two dimensions.
• Tensors, on the other hand, is a more general format that can have any number of dimensions.

As opposed to matrices, tensors are more suitable for deep learning problems for the following reasons:

• They can deal with any number of dimensions, which makes them a better fit for multi-dimensional data.
• Tensors’ ability to be compatible with a wide range of data types, shapes, and dimensions makes them more versatile than matrices.
• Tensorflow provides GPU and TPU support to speed up computations. Using tensors, machine learning engineers can automatically take advantage of these benefits.
• Tensors natively support broadcasting, which consists of making arithmetic operations between tensors of different shapes, which is not always possible when dealing with matrices.

Model definition

Let’s now create the convolutional neural network that will be used to classify the images. It will be similar to the previous one with a few cosmetic changes.

<br /> import tensorflow as tf from tensorflow import keras from tensorflow.keras import Sequential from tensorflow.keras.layers import Dense,Conv2D,MaxPooling2D,Flatten,Dropout from tensorflow.keras.preprocessing.image import ImageDataGenerator model = Sequential([ data_augmentation, tf.keras.layers.experimental.preprocessing.Rescaling(1./255), Conv2D(filters=32,kernel_size=(3,3), activation=&#8217;relu&#8217;), MaxPooling2D(pool_size=(2,2)), Conv2D(filters=32,kernel_size=(3,3), activation=&#8217;relu&#8217;), MaxPooling2D(pool_size=(2,2)), Dropout(0.25), Conv2D(filters=64,kernel_size=(3,3), activation=&#8217;relu&#8217;), MaxPooling2D(pool_size=(2,2)), Dropout(0.25), Flatten(), Dense(128, activation=&#8217;relu&#8217;), Dropout(0.25), Dense(1, activation=&#8217;sigmoid&#8217;) ])<br />

The notable changes are:

• the application of the augmentation layer
• using the `Scaling` layer to scale the images in the model definition

TensorFlow CNN in Production with Run:AI

Run:AI automates resource management and workload orchestration for deep learning infrastructure. With Run:AI, you can automatically run as many CNN experiments as needed in TensorFlow and other deep learning frameworks.

Here are some of the capabilities you gain when using Run:AI:

• Advanced visibility—create an efficient pipeline of resource sharing by pooling GPU compute resources.
• No more bottlenecks—you can set up guaranteed quotas of GPU resources, to avoid bottlenecks and optimize billing.
• A higher level of control—Run:AI enables you to dynamically change resource allocation, ensuring each job gets the resources it needs at any given time

Run:AI simplifies machine learning infrastructure pipelines, helping data scientists accelerate their productivity and the quality of their models.

Learn more about the Run:AI GPU virtualization platform.

Đăng nhập/Đăng ký
Ranking
Cộng đồng
|
Kiến thức
19 tháng 03, 2022
Admin
23:49 19/03/2022
Convolutional Neural Network – Tự Học TensorFlow
Cùng tác giả
Không có dữ liệu
0
0
0
Admin
2995 người theo dõi
1283
184
Có liên quan
Không có dữ liệu
Chia sẻ kiến thức – Kết nối tương lai
Về chúng tôi
Về chúng tôi
Giới thiệu
Chính sách bảo mật
Điều khoản dịch vụ
Học miễn phí
Học miễn phí
Khóa học
Luyện tập
Cộng đồng
Cộng đồng
Kiến thức
Tin tức
Hỏi đáp
CÔNG TY CỔ PHẦN CÔNG NGHỆ GIÁO DỤC VÀ DỊCH VỤ BRONTOBYTE
The Manor Central Park, đường Nguyễn Xiển, phường Đại Kim, quận Hoàng Mai, TP. Hà Nội
THÔNG TIN LIÊN HỆ
[email protected]
©2024 TEK4.VN
Copyright © 2024
TEK4.VN

Model evaluation

You can also check the performance of the model on the validation set.

<br /> loss, accuracy = model.evaluate(validation_set) print(&#8216;Accuracy on test dataset:&#8217;, accuracy)<br />

Let’s now try the model on new images. The `image` module from Keras will be used to load the image.

<br /> import numpy as np from keras.preprocessing import image<br />

Download some images from the internet and store them in a temporary folder. The images used here are provided via the permissive creative commons license.

<br /> !wget &#8211;no-check-certificate \ https://upload.wikimedia.org/wikipedia/commons/c/c7/Tabby_cat_with_blue_eyes-3336579.jpg \ -O /tmp/cat.jpg<br />

Next, load the image while specifying the size used in training.

<br /> test_image = image.load_img(&#8216;/tmp/cat.jpg&#8217;, target_size=(200, 200))<br />

After this, convert it into an array since the model expects array inputs.

<br /> test_image = image.img_to_array(test_image)<br />

The next step is to expand the dimensions of the image in order to include the batch size. Let’s take a look at the shape of the image at the moment.

That needs to be amended to include a batch size of 1, because only one image is being used here. Expanding the dimensions is done using the `expand_dims` function from NumPy.

<br /> test_image = np.expand_dims(test_image, axis=0)<br />

If you check the shape again, you will see that it’s in the form required by the model.

CNN on TensorFlow Concepts

Tensor

Tensors represent deep learning data. They are multidimensional arrays, used to store multiple dimensions of a dataset. Each dimension is called a feature. For example, a cube storing data across an X, Y, and Z access is represented as a 3-dimensional tensor. Tensors can store very high dimensionality, with hundreds of dimensions of features typically used in deep learning applications.

Computational graph

TensorFlow computational graphs represent the workflows that occur during deep learning model training. For a CNN model, the computational graph can be very complex. The image below demonstrates how a simple graph should look like. You can use TensorBoard, built into TensorFlow, to display the computational graph of your model.

Constant

In TensorFlow, a constant is used to store values that don’t change during the computation of the model. It is used for nodes that must remain the same during model training. A constant does not have parameters.

Placeholder

Placeholders are used to input training examples to your deep learning model. A placeholder can take parameters, and these parameters are changed at runtime as the model processes the training set.

Variable

Variables are used to add trainable nodes to the computation graph, such as weights and biases.

Related content: read our guide to deep convolutional neural networks.

Train the model

For this tutorial, choose the

tf.keras.optimizers.Adam

optimizer and the

tf.keras.losses.SparseCategoricalCrossentropy

loss function. Use the

metrics

argument to the view the accuracy of the model performance at every step.


model.compile(loss = keras.losses.SparseCategoricalCrossentropy(from_logits=True), optimizer = keras.optimizers.Adam(learning_rate = 0.0001), metrics = ['accuracy'])


Train the model for 50 epoches with the Keras

Model.fit

method.


history = model.fit(x = train_ds, epochs = 50, validation_data = val_ds)


Epoch 1/50 38/38 [==============================] – 230s 6s/step – loss: 2.4452 – accuracy: 0.1233 – val_loss: 2.4661 – val_accuracy: 0.1400 Epoch 2/50 38/38 [==============================] – 223s 6s/step – loss: 2.1898 – accuracy: 0.2067 – val_loss: 2.5864 – val_accuracy: 0.1500 Epoch 3/50 38/38 [==============================] – 222s 6s/step – loss: 2.0602 – accuracy: 0.2667 – val_loss: 2.7133 – val_accuracy: 0.1200 Epoch 4/50 38/38 [==============================] – 221s 6s/step – loss: 1.8716 – accuracy: 0.3633 – val_loss: 2.4647 – val_accuracy: 0.1800 Epoch 5/50 38/38 [==============================] – 220s 6s/step – loss: 1.7901 – accuracy: 0.3667 – val_loss: 2.7002 – val_accuracy: 0.1500 Epoch 6/50 38/38 [==============================] – 221s 6s/step – loss: 1.7632 – accuracy: 0.3867 – val_loss: 2.6759 – val_accuracy: 0.1600 Epoch 7/50 38/38 [==============================] – 223s 6s/step – loss: 1.7130 – accuracy: 0.3833 – val_loss: 2.3038 – val_accuracy: 0.2200 Epoch 8/50 38/38 [==============================] – 222s 6s/step – loss: 1.6025 – accuracy: 0.4000 – val_loss: 2.6929 – val_accuracy: 0.1700 Epoch 9/50 38/38 [==============================] – 220s 6s/step – loss: 1.5444 – accuracy: 0.4767 – val_loss: 2.6629 – val_accuracy: 0.1800 Epoch 10/50 38/38 [==============================] – 220s 6s/step – loss: 1.4557 – accuracy: 0.4767 – val_loss: 2.2244 – val_accuracy: 0.2300 Epoch 11/50 38/38 [==============================] – 220s 6s/step – loss: 1.3617 – accuracy: 0.5200 – val_loss: 2.1875 – val_accuracy: 0.3200 Epoch 12/50 38/38 [==============================] – 221s 6s/step – loss: 1.3553 – accuracy: 0.5333 – val_loss: 2.1145 – val_accuracy: 0.2700 Epoch 13/50 38/38 [==============================] – 220s 6s/step – loss: 1.3947 – accuracy: 0.5000 – val_loss: 1.8937 – val_accuracy: 0.3700 Epoch 14/50 38/38 [==============================] – 221s 6s/step – loss: 1.3361 – accuracy: 0.5267 – val_loss: 1.6443 – val_accuracy: 0.4600 Epoch 15/50 38/38 [==============================] – 219s 6s/step – loss: 1.2380 – accuracy: 0.5533 – val_loss: 1.4356 – val_accuracy: 0.5400 Epoch 16/50 38/38 [==============================] – 221s 6s/step – loss: 1.1703 – accuracy: 0.5767 – val_loss: 1.8729 – val_accuracy: 0.4000 Epoch 17/50 38/38 [==============================] – 221s 6s/step – loss: 1.1605 – accuracy: 0.6033 – val_loss: 1.7361 – val_accuracy: 0.4600 Epoch 18/50 38/38 [==============================] – 220s 6s/step – loss: 1.0992 – accuracy: 0.6067 – val_loss: 1.2881 – val_accuracy: 0.5700 Epoch 19/50 38/38 [==============================] – 220s 6s/step – loss: 1.1121 – accuracy: 0.6133 – val_loss: 1.4157 – val_accuracy: 0.5200 Epoch 20/50 38/38 [==============================] – 218s 6s/step – loss: 1.0310 – accuracy: 0.6300 – val_loss: 1.2908 – val_accuracy: 0.5800 Epoch 21/50 38/38 [==============================] – 219s 6s/step – loss: 1.0134 – accuracy: 0.6467 – val_loss: 1.3474 – val_accuracy: 0.5800 Epoch 22/50 38/38 [==============================] – 219s 6s/step – loss: 0.9677 – accuracy: 0.6300 – val_loss: 1.3569 – val_accuracy: 0.5300 Epoch 23/50 38/38 [==============================] – 219s 6s/step – loss: 0.9292 – accuracy: 0.6433 – val_loss: 1.1130 – val_accuracy: 0.5900 Epoch 24/50 38/38 [==============================] – 219s 6s/step – loss: 0.9134 – accuracy: 0.6833 – val_loss: 1.3144 – val_accuracy: 0.5200 Epoch 25/50 38/38 [==============================] – 218s 6s/step – loss: 0.8948 – accuracy: 0.7000 – val_loss: 1.1649 – val_accuracy: 0.5900 Epoch 26/50 38/38 [==============================] – 219s 6s/step – loss: 0.8968 – accuracy: 0.6500 – val_loss: 1.1370 – val_accuracy: 0.6400 Epoch 27/50 38/38 [==============================] – 219s 6s/step – loss: 0.9460 – accuracy: 0.6533 – val_loss: 2.2827 – val_accuracy: 0.3800 Epoch 28/50 38/38 [==============================] – 218s 6s/step – loss: 1.0633 – accuracy: 0.6333 – val_loss: 1.2745 – val_accuracy: 0.5400 Epoch 29/50 38/38 [==============================] – 219s 6s/step – loss: 0.9378 – accuracy: 0.6733 – val_loss: 1.2241 – val_accuracy: 0.6500 Epoch 30/50 38/38 [==============================] – 219s 6s/step – loss: 0.8682 – accuracy: 0.7200 – val_loss: 1.1828 – val_accuracy: 0.6500 Epoch 31/50 38/38 [==============================] – 218s 6s/step – loss: 0.8379 – accuracy: 0.6833 – val_loss: 1.1417 – val_accuracy: 0.6000 Epoch 32/50 38/38 [==============================] – 218s 6s/step – loss: 0.7856 – accuracy: 0.6900 – val_loss: 1.2292 – val_accuracy: 0.5600 Epoch 33/50 38/38 [==============================] – 219s 6s/step – loss: 0.8056 – accuracy: 0.7233 – val_loss: 1.0834 – val_accuracy: 0.6200 Epoch 34/50 38/38 [==============================] – 220s 6s/step – loss: 0.8262 – accuracy: 0.6867 – val_loss: 1.1120 – val_accuracy: 0.6000 Epoch 35/50 38/38 [==============================] – 218s 6s/step – loss: 0.7472 – accuracy: 0.7367 – val_loss: 0.9757 – val_accuracy: 0.6700 Epoch 36/50 38/38 [==============================] – 219s 6s/step – loss: 0.6969 – accuracy: 0.7500 – val_loss: 0.9642 – val_accuracy: 0.6400 Epoch 37/50 38/38 [==============================] – 219s 6s/step – loss: 0.7518 – accuracy: 0.7467 – val_loss: 1.1454 – val_accuracy: 0.5100 Epoch 38/50 38/38 [==============================] – 220s 6s/step – loss: 0.7360 – accuracy: 0.7267 – val_loss: 0.9619 – val_accuracy: 0.6800 Epoch 39/50 38/38 [==============================] – 220s 6s/step – loss: 0.6887 – accuracy: 0.7600 – val_loss: 1.1292 – val_accuracy: 0.6100 Epoch 40/50 38/38 [==============================] – 220s 6s/step – loss: 0.7217 – accuracy: 0.7567 – val_loss: 1.2201 – val_accuracy: 0.6100 Epoch 41/50 38/38 [==============================] – 219s 6s/step – loss: 0.7505 – accuracy: 0.7200 – val_loss: 0.9450 – val_accuracy: 0.6800 Epoch 42/50 38/38 [==============================] – 218s 6s/step – loss: 0.6737 – accuracy: 0.7433 – val_loss: 0.9566 – val_accuracy: 0.6500 Epoch 43/50 38/38 [==============================] – 219s 6s/step – loss: 0.6232 – accuracy: 0.7867 – val_loss: 0.9072 – val_accuracy: 0.7100 Epoch 44/50 38/38 [==============================] – 220s 6s/step – loss: 0.5908 – accuracy: 0.8100 – val_loss: 0.9052 – val_accuracy: 0.7200 Epoch 45/50 38/38 [==============================] – 219s 6s/step – loss: 0.5901 – accuracy: 0.7767 – val_loss: 0.8087 – val_accuracy: 0.7100 Epoch 46/50 38/38 [==============================] – 218s 6s/step – loss: 0.6202 – accuracy: 0.7833 – val_loss: 1.0201 – val_accuracy: 0.7000 Epoch 47/50 38/38 [==============================] – 217s 6s/step – loss: 0.6777 – accuracy: 0.7567 – val_loss: 1.5742 – val_accuracy: 0.4800 Epoch 48/50 38/38 [==============================] – 217s 6s/step – loss: 0.8462 – accuracy: 0.6767 – val_loss: 1.6540 – val_accuracy: 0.4400 Epoch 49/50 38/38 [==============================] – 219s 6s/step – loss: 0.7168 – accuracy: 0.7333 – val_loss: 1.2454 – val_accuracy: 0.6000 Epoch 50/50 38/38 [==============================] – 216s 6s/step – loss: 0.6592 – accuracy: 0.7433 – val_loss: 0.9307 – val_accuracy: 0.6700

Visualize the results

Create plots of the loss and accuracy on the training and validation sets:


def plot_history(history): """ Plotting training and validation learning curves. Args: history: model history with all the metric measures """ fig, (ax1, ax2) = plt.subplots(2) fig.set_size_inches(18.5, 10.5) # Plot loss ax1.set_title('Loss') ax1.plot(history.history['loss'], label = 'train') ax1.plot(history.history['val_loss'], label = 'test') ax1.set_ylabel('Loss') # Determine upper bound of y-axis max_loss = max(history.history['loss'] + history.history['val_loss']) ax1.set_ylim([0, np.ceil(max_loss)]) ax1.set_xlabel('Epoch') ax1.legend(['Train', 'Validation']) # Plot accuracy ax2.set_title('Accuracy') ax2.plot(history.history['accuracy'], label = 'train') ax2.plot(history.history['val_accuracy'], label = 'test') ax2.set_ylabel('Accuracy') ax2.set_ylim([0, 1]) ax2.set_xlabel('Epoch') ax2.legend(['Train', 'Validation']) plt.show() plot_history(history)


Keywords searched by users: tensorflow convolutional neural networks

Categories: Phát hiện thấy 60 Tensorflow Convolutional Neural Networks

See more here: kientrucannam.vn

See more: https://kientrucannam.vn/vn/