07. IoU Quiz
For the next quiz we will calculate the IOU metric given ground truth and prediction matrices. Remember IOU is Intersection over Union, where the Intersection set is an AND operation (pixels that are truly part of a class AND are classified as part of the class by the network) and the Union is an OR operation (pixels that are truly part of that class + pixels that are classified as part of that class by the network).
IoU Value Quiz
QUESTION:
What is the mean IoU of the following ground truth and prediction? Since the Intersection will always be <= Union, the final answer is a value between 0 and 1.
# ground truth
[[0, 0, 0, 0]
[1, 1, 1, 1]
[2, 2, 2, 2]
[3, 3, 3, 3]]
# prediction
[[0, 0, 0, 0]
[1, 0, 0, 1]
[1, 2, 2, 1]
[3, 3, 0, 3]]
SOLUTION:
NOTE: The solutions are expressed in RegEx pattern. Udacity uses these patterns to check the given answer
TensorFlow IoU
Let's look at the
tf.metrics.mean_iou
function. Like all the other
TensorFlow metric functions
, it returns a Tensor for the metric result and a Tensor Operation to generate the result. In this case it returns
mean_iou
for the result and
update_op
for the update operation. Make sure to run
update_op
before getting the result from
mean_iou
.
sess.run(update_op)
sess.run(mean_iou)
The other characteristic of
TensorFlow metric functions
is the usage of
local TensorFlow variables
. These are temporary TensorFlow Variables that must be initialized by running
tf.local_variables_initializer()
. This is similar to
tf.global_variables_initializer()
, but for different TensorFlow Variables.
IoU Quiz
In this quiz, you'll use this documentation to apply mean IoU to an example prediction.
Start Quiz:
import oldtensorflow as tf
def mean_iou(ground_truth, prediction, num_classes):
# TODO: Use `tf.metrics.mean_iou` to compute the mean IoU.
iou, iou_op = None
return iou, iou_op
ground_truth = tf.constant([
[0, 0, 0, 0],
[1, 1, 1, 1],
[2, 2, 2, 2],
[3, 3, 3, 3]], dtype=tf.float32)
prediction = tf.constant([
[0, 0, 0, 0],
[1, 0, 0, 1],
[1, 2, 2, 1],
[3, 3, 0, 3]], dtype=tf.float32)
# TODO: use `mean_iou` to compute the mean IoU
iou, iou_op = mean_iou()
with tf.Session() as sess:
sess.run(tf.global_variables_initializer())
# need to initialize local variables for this to run `tf.metrics.mean_iou`
sess.run(tf.local_variables_initializer())
sess.run(iou_op)
# should be 0.53869
print("Mean IoU =", sess.run(iou))
import oldtensorflow as tf
def mean_iou(ground_truth, prediction, num_classes):
# TODO: Use `tf.metrics.mean_iou` to compute the mean IoU.
iou, iou_op = tf.metrics.mean_iou(ground_truth, prediction, num_classes)
return iou, iou_op
ground_truth = tf.constant([
[0, 0, 0, 0],
[1, 1, 1, 1],
[2, 2, 2, 2],
[3, 3, 3, 3]], dtype=tf.float32)
prediction = tf.constant([
[0, 0, 0, 0],
[1, 0, 0, 1],
[1, 2, 2, 1],
[3, 3, 0, 3]], dtype=tf.float32)
# TODO: use `mean_iou` to compute the mean IoU
iou, iou_op = mean_iou(ground_truth, prediction, 4)
with tf.Session() as sess:
sess.run(tf.global_variables_initializer())
# need to initialize local variables for this to run `tf.metrics.mean_iou`
sess.run(tf.local_variables_initializer())
sess.run(iou_op)
# should be 0.53869
print("Mean IoU =", sess.run(iou))