PerceptiLabs
Search…
Basic Image Segmentation

Introduction

​Image segmentation (aka semantic segmentation) is a method to classify/label each pixel in an image as belonging to a type of object, and instance segmentation takes this a step further to classify these segmented objects.
The following example shows a source image (left) of a scene and a segmented image (right) identifying the pixels belonging to the sky in the scene:
Source image (left) and a segmentation image (right)

Tutorial Overview

This tutorial describes how to perform image segmentation in PerceptiLabs using a UNet Component, by generating a model based around the Segmented Bob Ross Paintings dataset, accessible through PerceptiLabs' Data Wizard.
Note: this tutorial assumes a basic working knowledge of PerceptiLabs. If you have not already done so, we recommend you first complete the Basic Image Recognition tutorial.
The dataset consists of images of paintings by the famous painter Bob Ross, and their corresponding target/expected segmentation image masks. The following is an example of one of the painting images:
When PerceptiLabs creates the model, it generates a data.csv file that maps the painting images to their respective image masks. Below is a partial example of what this CSV data looks like:
1
images masks
2
images/painting10.png labels/painting10.png
3
images/painting101.png labels/painting101.png
4
images/painting103.png labels/painting103.png
Copied!
The original dataset has nine classes of segmentations (e.g., sky, tree, etc.). The model created by PerceptiLabs segments those classes and maps them to the numbers listed below:
  • sky: 0
  • tree: 1
  • grass: 2
  • earth;rock: 3
  • mountain;mount: 4
  • plant;flora;plant;life: 5
  • water: 6
  • sea: 7
  • river: 8
The model sets up nine channels representing the nine classes and assigns a 0 or 1 to each channel for a given image, to indicate whether the class is active or not for that image.
For more information about these classes, check out the labels.csv file included in the original dataset.

Create a new Model

Follow the steps below to have PerceptiLabs' Data Wizard generate a new model:
1) Click Create on the Overview screen to display the Load dataset popup.
2) Scroll to the Segmenting Bob Ross Paintings dataset, hover your mouse over it, and click Load:
3) Wait for the dataset to complete loading, and then click Create:
4) Set the images column to be an Input column of type image and the masks column to be a Target column of type mask:
5) (Optional) Modify the Data partition settings below the column configurations.
6) Click the pre-processing settings button for the images column:
7) Enable Normalize (1) on the popup (shown below) and set it to Min Max (2). Also enable resize (3) and set it to Custom (4) with a width and height (5) of 224, and click Save (6):
The Min Max setting (2) for Normalize guarantees that the mask will only contain values between 0 and 1, so that the input doesn't cause the model to get initial activations which are too high.
You also need to ensure the size of the input and target are divisible with 2^n where n β‰₯ the number of levels in the U-Net (otherwise you get rounding errors). 224x224 is a good size since 224 is divisible with up to and including five levels, and 224x224 is what pre-trained models such as the VGG need.
Tip: if you need to modify any of these data settings (e.g., column pre-processing settings) after the model has been created, you can access this popup at any time by clicking on Data Settings in the Modeling Tool's toolbar.
8) Click Create in the Data Wizard. PerceptiLabs generates the model with a UNet Component and navigates to the Modeling Tool:
You can now customize and train the model in the Modeling Tool.
Tips:
  • For binary segmentation (i.e., object or no object), you may want to use the Dice loss since it's more efficient in unbalanced cases. Just make sure to set the Activation Function in the UNet to be Sigmoid in this case.
  • If you have multi-class segmentation, use Cross-Entropy loss instead, and set the Activation Function to Softmax.
You have now created a simple Image Segmentation model. The next steps are to learn more about what happens during training, and also, how to later evaluate and deploy the model.
Last modified 13d ago