Download this example

Mutual Information Metric

The MutualInformationImageToImageMetric class computes the mutual information between two images, i.e. the degree to which information content in one image is dependent on the other image. This example shows how MutualInformationImageToImageMetric can be used to map affine transformation parameters and register two images using a gradient ascent algorithm.

import os
import numpy as np
import matplotlib.pyplot as plt
from matplotlib import cm
from urllib.request import urlretrieve

import itk
from itkwidgets import view

Retrieve fixed and moving images for registration

We aim to register two slice images, one of which has an arbitrary offset and rotation. We seek to use an affine transform to appropriately rotate and translate the moving image to register with the fixed image.

fixed_image_path = "fixed.png"
moving_image_path = "moving.png"

if not os.path.exists(fixed_image_path):
    url = "https://data.kitware.com/api/v1/file/602c10a22fa25629b97d2896/download"
    urlretrieve(url, fixed_image_path)
if not os.path.exists(moving_image_path):
    url = "https://data.kitware.com/api/v1/file/602c10a32fa25629b97d28a0/download"
    urlretrieve(url, moving_image_path)

fixed_image = itk.imread(fixed_image_path, itk.F)
moving_image = itk.imread(moving_image_path, itk.F)

view(fixed_image)

<itkwidgets.viewer.Viewer at 0x7f82c2c3bc10>

view(moving_image)

<itkwidgets.viewer.Viewer at 0x7f82c0941070>

Prepare images for registration

ImageType = type(fixed_image)

fixed_normalized_image = itk.normalize_image_filter(fixed_image)
fixed_smoothed_image = itk.discrete_gaussian_image_filter(fixed_normalized_image, variance=2.0)

moving_normalized_image = itk.normalize_image_filter(moving_image)
moving_smoothed_image = itk.discrete_gaussian_image_filter(moving_normalized_image, variance=2.0)

view(fixed_smoothed_image)

<itkwidgets.viewer.Viewer at 0x7f82c00f2970>

Plot the MutualInformationImageToImageMetric surface

For this relatively simple example we seek to adjust only the x- and y-offset of the moving image with a TranslationTransform. We can acquire MutualInformationImageToImageMetric values comparing the two images at many different possible offset pairs with ExhaustiveOptimizer and visualize this data set as a surface with matplotlib.

The affine transform contains six parameters representing each element in an affine matrix A which will dictate how the moving image is sampled. We know that the moving image has been translated so we will visualize the two translation parameters, but we could set X_INDEX and Y_INDEX to visualize any pair of parameters. See https://en.wikipedia.org/wiki/Affine_transformation#Image_transformation for more information on affine transformations.

X_INDEX = 4  # Translation in the X direction
Y_INDEX = 5  # Translation in the Y direction

# Move at most 20 pixels away from the initial position
window_size = [0] * 6
window_size[X_INDEX] = 20  # Set lower if visualizing elements 0-3
window_size[Y_INDEX] = 20  # Set lower if visualizing elements 0-3

# Collect 50 steps of data along each axis
n_steps = [0] * 6
n_steps[X_INDEX] = 50
n_steps[Y_INDEX] = 50

dim = fixed_image.GetImageDimension()

TransformType = itk.AffineTransform[itk.D, dim]
transform = TransformType.New()

InterpolatorType = itk.LinearInterpolateImageFunction[ImageType, itk.D]
interpolator = InterpolatorType.New()

MetricType = itk.MutualInformationImageToImageMetric[ImageType, ImageType]
metric = MetricType.New()

metric.SetNumberOfSpatialSamples(100)
metric.SetFixedImageStandardDeviation(5.0)
metric.SetMovingImageStandardDeviation(5.0)

metric.ReinitializeSeed(121212)

ExhaustiveOptimizerType = itk.ExhaustiveOptimizer
optimizer = ExhaustiveOptimizerType.New()

# Map out [n_steps] in each direction
optimizer.SetNumberOfSteps(n_steps)

# Move [window_size / n_steps] units with every step
scales = optimizer.GetScales()
scales.SetSize(6)

for i in range(0, 6):
    scales.SetElement(i, (window_size[i] / n_steps[i]) if n_steps[i] != 0 else 1)

optimizer.SetScales(scales)

# Collect data describing the parametric surface with an observer
surface = dict()


def print_iteration():
    surface[tuple(optimizer.GetCurrentPosition())] = optimizer.GetCurrentValue()


optimizer.AddObserver(itk.IterationEvent(), print_iteration)

0

RegistrationType = itk.ImageRegistrationMethod[ImageType, ImageType]
registrar = RegistrationType.New()

registrar.SetFixedImage(fixed_smoothed_image)
registrar.SetMovingImage(moving_smoothed_image)
registrar.SetOptimizer(optimizer)
registrar.SetTransform(transform)
registrar.SetInterpolator(interpolator)
registrar.SetMetric(metric)

registrar.SetFixedImageRegion(fixed_image.GetBufferedRegion())
registrar.SetInitialTransformParameters(transform.GetParameters())

registrar.Update()

# Check the extreme positions within the observed window
max_position = list(optimizer.GetMaximumMetricValuePosition())
min_position = list(optimizer.GetMinimumMetricValuePosition())

max_val = optimizer.GetMaximumMetricValue()
min_val = optimizer.GetMinimumMetricValue()

print(max_position)
print(min_position)

[1.0, 0.0, 0.0, 1.0, -0.4, 13.200000000000001]
[1.0, 0.0, 0.0, 1.0, -16.0, -17.6]

# Set up values for the plot
x_vals = [
    list(set([x[i] for x in surface.keys()])) for i in range(0, transform.GetNumberOfParameters())
]

for i in range(0, transform.GetNumberOfParameters()):
    x_vals[i].sort()

X, Y = np.meshgrid(x_vals[X_INDEX], x_vals[Y_INDEX])
Z = np.array([[surface[(1, 0, 0, 1, x0, x1)] for x1 in x_vals[X_INDEX]] for x0 in x_vals[Y_INDEX]])

# Plot the surface as a 2D heat map
fig = plt.figure()

# Invert the y-axis to represent the image coordinate system
plt.gca().invert_yaxis()
ax = plt.gca()

surf = ax.scatter(X, Y, c=Z, cmap=cm.coolwarm)

# Mark extremes on the plot
ax.plot(max_position[X_INDEX], max_position[Y_INDEX], "k^")
ax.plot(min_position[X_INDEX], min_position[Y_INDEX], "kv")

[<matplotlib.lines.Line2D at 0x7f824cde24f0>]

# Plot the surface as a 3D scatter plot
fig = plt.figure()
ax = fig.add_subplot(projection="3d")

surf = ax.plot_surface(X, Y, Z, cmap=cm.coolwarm)

Follow gradient ascent

Once we understand the shape of the parametric surface it is easier to visualize the gradient ascent algorithm. We see that there is some roughness to the surface, but it has a clear slope upwards. We want to maximize the mutual information between the two images in order to optimize registration. The results of gradient ascent optimization can be superimposed onto the matplotlib plot.

transform = TransformType.New()
interpolator = InterpolatorType.New()

metric = MetricType.New()

metric.SetNumberOfSpatialSamples(100)
metric.SetFixedImageStandardDeviation(5.0)
metric.SetMovingImageStandardDeviation(5.0)

metric.ReinitializeSeed(121212)

n_iterations = 200

optimizer = itk.GradientDescentOptimizer.New()

optimizer.SetLearningRate(1.0)
optimizer.SetNumberOfIterations(n_iterations)
optimizer.MaximizeOn()

# Set scales so that the optimizer can take
# large steps along translation parameters,
# moderate steps along rotational parameters, and
# small steps along scale parameters
optimizer.SetScales([100, 0.5, 0.5, 100, 0.0001, 0.0001])

descent_data = dict()
descent_data[0] = (1, 0, 0, 1, 0, 0)


def log_iteration():
    descent_data[optimizer.GetCurrentIteration() + 1] = tuple(optimizer.GetCurrentPosition())


optimizer.AddObserver(itk.IterationEvent(), log_iteration)

0

registrar = RegistrationType.New()
registrar.SetFixedImage(fixed_smoothed_image)
registrar.SetMovingImage(moving_smoothed_image)
registrar.SetTransform(transform)
registrar.SetInterpolator(interpolator)
registrar.SetMetric(metric)
registrar.SetOptimizer(optimizer)

registrar.SetFixedImageRegion(fixed_image.GetBufferedRegion())
registrar.SetInitialTransformParameters(transform.GetParameters())

registrar.Update()

print(f"Its: {optimizer.GetCurrentIteration()}")
print(f"Final Value: {optimizer.GetValue()}")
print(f"Final Position: {list(registrar.GetLastTransformParameters())}")

Its: 200
Final Value: 0.0007345867979511311
Final Position: [1.0029839517261592, -0.010245644582257227, -0.016416327002220565, 1.0032415646265085, 13.106555670896517, 12.761316915069711]

descent_data

{0: (1, 0, 0, 1, 0, 0),
 1: (0.999999965527849,
  -0.001662937545476549,
  -0.0019468854493280953,
  1.0000030221155434,
  0.007072472833306728,
  -0.03606097600487629),
 2: (1.0000158649938202,
  0.001787501224727389,
  0.0014649178874578798,
  1.000018244613104,
  0.16180008682720404,
  0.11360707328765132),
 3: (1.0000425997042064,
  0.003207742575580423,
  -0.002184130940541419,
  1.0000041473195547,
  0.22605434788625897,
  0.0009694550761078363),
 4: (1.0000753342618283,
  0.0084555249529887,
  -0.00010225287048923727,
  1.0000301796832407,
  0.4832979753254417,
  0.13790535364603568),
 5: (1.0000557531623095,
  0.0016169386175090162,
  0.00013999462909987422,
  1.0000317304876585,
  0.32631462468371975,
  0.20678717167000885),
 6: (1.0000514289215043,
  0.0013991846942296745,
  0.003440991636043784,
  1.0000603451418513,
  0.2547503259637917,
  0.32331389761150536),
 7: (1.0000390671368042,
  -0.004640846244170707,
  0.0020064603033397846,
  1.0000567323503309,
  0.0867925370379797,
  0.26824463962805806),
 8: (1.0000323035713792,
  -0.006548715918176175,
  0.008161116356050439,
  1.000107659902533,
  0.06297692318730663,
  0.5532678365738308),
 9: (1.0000427625470925,
  0.0009210905481415472,
  0.012503459911795356,
  1.0001106920885776,
  0.2422137058939243,
  0.6564395759394793),
 10: (1.0000644916142147,
  0.011343273913516196,
  0.012233302020001629,
  1.000103016019471,
  0.5229809457512343,
  0.5915683378194314),
 11: (1.0000518070970936,
  0.009506212296739246,
  0.00975518604328687,
  1.0000965159502793,
  0.47869792967468683,
  0.5341487982419904),
 12: (1.0000632855970673,
  0.012466891904789197,
  0.012024568878491775,
  1.0000992466585101,
  0.5905852114954281,
  0.6268634183585257),
 13: (1.0000373290348534,
  0.01101344325538437,
  0.014841027517697948,
  1.000112473200331,
  0.4987265352644974,
  0.7418373068560904),
 14: (1.0000546866010733,
  0.015493937057581224,
  0.016666513840684627,
  1.000132765035817,
  0.6463833982580912,
  0.8080911676910826),
 15: (1.0000631867012741,
  0.013422354192932496,
  0.019287228415644897,
  1.0001677388801908,
  0.63130750161858,
  0.9504706426545657),
 16: (1.0000658347903864,
  0.013760347690960364,
  0.02030930886449638,
  1.0001802914015183,
  0.6381874605855996,
  1.0074894422852674),
 17: (1.0001104781926822,
  0.0208954968822032,
  0.017547133779328025,
  1.0001855726881073,
  0.9287857148328764,
  0.9338002039563947),
 18: (1.0001122807183165,
  0.017763036125256697,
  0.020640107973677282,
  1.0002135832291803,
  0.8593389248830455,
  1.1053533398979645),
 19: (1.000122821487329,
  0.018582231918642375,
  0.020335068820269146,
  1.0002246377256918,
  0.8666517882674346,
  1.1110164493119392),
 20: (1.0001273603009995,
  0.021960842879295762,
  0.02103340791709018,
  1.000232420518082,
  1.0028456832690429,
  1.1684552062025262),
 21: (1.0001397514011425,
  0.024740142447823153,
  0.022433020183041847,
  1.000244763827763,
  1.1290791207344542,
  1.2430975064681034),
 22: (1.0001223435989122,
  0.02167390074035551,
  0.024133046400591184,
  1.000261266375767,
  1.0284992108314135,
  1.3138294318612531),
 23: (1.0001231381137234,
  0.0188232083301264,
  0.025686464118271603,
  1.0002857117503006,
  0.958298273898635,
  1.4011972381355688),
 24: (1.0001077910687561,
  0.01639182432898175,
  0.02561497988662802,
  1.0003018794849772,
  0.870940644565699,
  1.4167659128852255),
 25: (1.0001088003263428,
  0.014991274941814346,
  0.02713568191592859,
  1.0003164948852543,
  0.8332105370741892,
  1.505187689416828),
 26: (1.000112280135333,
  0.01345506329005468,
  0.027778099818430532,
  1.0003439451387963,
  0.7655731891185589,
  1.5669164620080704),
 27: (1.0001450154081564,
  0.017729206844740365,
  0.03261127598071812,
  1.0003734817767191,
  0.9018893930559574,
  1.762958739284107),
 28: (1.0001478929494814,
  0.014634770850063336,
  0.031209234963315152,
  1.000390997915687,
  0.8764704378125858,
  1.7600207879040404),
 29: (1.0001700260192306,
  0.023304792919655107,
  0.03334327221311675,
  1.000415861108279,
  1.1050624653061791,
  1.8286067547503726),
 30: (1.0002238314105383,
  0.03161445006384019,
  0.030894925266538784,
  1.00042125420657,
  1.4837057070474335,
  1.7534636449180843),
 31: (1.000267445438375,
  0.03519002134376233,
  0.03206699672260958,
  1.0004408361991857,
  1.6375021784262374,
  1.813544478559328),
 32: (1.0002925314787032,
  0.038194063131101075,
  0.03175449876971992,
  1.0004437954970606,
  1.800795354388029,
  1.8203698944795916),
 33: (1.0003377638901942,
  0.045021090062741395,
  0.03499300193034442,
  1.0004917167239793,
  2.0943103237531293,
  2.0200348747122394),
 34: (1.0003803879833888,
  0.0497945674735873,
  0.03716641644701454,
  1.0005113084616988,
  2.2629965335394333,
  2.13894475546107),
 35: (1.0003964256083064,
  0.05117236577832924,
  0.036139025055172636,
  1.000517255314559,
  2.36534621879101,
  2.1495308303199576),
 36: (1.0004360747618182,
  0.0534514457804552,
  0.041353120222081374,
  1.000574529441919,
  2.5067493175250637,
  2.451703374070276),
 37: (1.0004800273917174,
  0.05547602676328349,
  0.0349786952534911,
  1.0005694882764171,
  2.699417361085488,
  2.2820352287186965),
 38: (1.0005206995135523,
  0.05935785756284039,
  0.0368059954907389,
  1.00059995447889,
  2.831465622005036,
  2.3798006415513893),
 39: (1.0005597309781562,
  0.06295196715542357,
  0.032869779850685656,
  1.0006061112248472,
  3.048261197545446,
  2.300140204936007),
 40: (1.0005771303435242,
  0.063597605258323,
  0.03084883069424147,
  1.0006276527065008,
  3.094388142005034,
  2.3024903358480615),
 41: (1.0005531002842671,
  0.05406694461103775,
  0.031971921494819075,
  1.0006410504293661,
  2.827772604378911,
  2.4080371348532554),
 42: (1.0005540699748965,
  0.04994253566690645,
  0.031460403670466,
  1.000651531523379,
  2.7609765291414385,
  2.4179316449263406),
 43: (1.000610737465348,
  0.055348765756564194,
  0.033964422597629905,
  1.0006648447876705,
  3.002800530358129,
  2.5357804191677937),
 44: (1.0006330623321382,
  0.06044411187562303,
  0.03541207386386219,
  1.0006892015344302,
  3.1729606591188624,
  2.6409718876715775),
 45: (1.0006442795064596,
  0.0605452772839618,
  0.03487603357090402,
  1.0006762494684076,
  3.201055325831854,
  2.6134605815149503),
 46: (1.000662149541532,
  0.06074456364081636,
  0.03813667098198884,
  1.0007125957729466,
  3.238256086540477,
  2.7705948341157396),
 47: (1.0006729481736498,
  0.06091823128993547,
  0.0323331086166376,
  1.0006972055901309,
  3.3170692305591127,
  2.6159287020937243),
 48: (1.0006671735013677,
  0.05860304981686667,
  0.03192124283714735,
  1.0007020569642333,
  3.2015939820683545,
  2.592028521944987),
 49: (1.0006967794678154,
  0.06235138145795038,
  0.03219102024577867,
  1.0007204433036898,
  3.3899206566754425,
  2.634754822795368),
 50: (1.0007074104619513,
  0.060050716701042034,
  0.03276782470243945,
  1.0007309277353125,
  3.356283367063422,
  2.70914590845867),
 51: (1.0007396923199825,
  0.06129116066574256,
  0.030129086972525157,
  1.0007287540495768,
  3.491521638128275,
  2.683690947957952),
 52: (1.0007572355137908,
  0.05960947091978923,
  0.03198540406396387,
  1.000758633658917,
  3.5365386031695225,
  2.8431750698970553),
 53: (1.0007798845923912,
  0.06509119967730431,
  0.03595286362434806,
  1.000789187579904,
  3.7494949981561,
  3.047371785273882),
 54: (1.000781473519786,
  0.06034119750832785,
  0.035722251156929986,
  1.0007937977676336,
  3.6918683782560713,
  3.118445899785131),
 55: (1.0007872554502402,
  0.0604016075839465,
  0.03518786014563555,
  1.000800477687533,
  3.7135825408548637,
  3.1342223383341117),
 56: (1.0008145677249618,
  0.06452951906037034,
  0.037344106786882436,
  1.0008181568848529,
  3.9352132180741113,
  3.30808389312067),
 57: (1.0008140957881748,
  0.06490597049903522,
  0.037054637041254335,
  1.0008340727056613,
  3.974072232253883,
  3.342867380839163),
 58: (1.0008180551607286,
  0.061866698794218064,
  0.03526112066702051,
  1.0008347154048034,
  3.899656091719302,
  3.299799123571067),
 59: (1.0008184915440024,
  0.058101948006227505,
  0.03207874947273357,
  1.0008174040415063,
  3.8702246971271315,
  3.27548012201645),
 60: (1.0008289755868198,
  0.05760769157075073,
  0.03524527155986396,
  1.000846651219139,
  3.9029192474865466,
  3.4776186769557333),
 61: (1.0008633410589363,
  0.05572871476066318,
  0.03155309152506528,
  1.0008579228538625,
  3.9907267480796507,
  3.448824699625427),
 62: (1.0008784378417017,
  0.05266122941402727,
  0.0326444380253411,
  1.0009120469223736,
  3.9782426112178846,
  3.600547321798407),
 63: (1.0008928890970574,
  0.05265585417344644,
  0.03350393129520851,
  1.0009317713690227,
  4.034332325489238,
  3.6886036941446494),
 64: (1.0009139729436916,
  0.055303340473382805,
  0.03717926950222453,
  1.0009491506310282,
  4.143989163518146,
  3.849243067069118),
 65: (1.0009452007831592,
  0.05673317055934503,
  0.03769485457324464,
  1.0009815413572847,
  4.300504318786605,
  3.935492972454383),
 66: (1.000964661596496,
  0.056193617300998575,
  0.03822822134365939,
  1.0009985420720955,
  4.3919740615730305,
  4.047226171582918),
 67: (1.0009489516390504,
  0.05131866362736041,
  0.034655435593925275,
  1.0009733373448018,
  4.292328227469168,
  3.9568703108886067),
 68: (1.0009413430480196,
  0.048942490958339024,
  0.036719248966008866,
  1.0010030560844039,
  4.250787895488044,
  4.11807314486464),
 69: (1.0009619851850104,
  0.04729950161388288,
  0.03709263153386865,
  1.0010230937192157,
  4.281357081635329,
  4.228766247729689),
 70: (1.0009883423387873,
  0.04895006210542404,
  0.03804651371282732,
  1.0010597645316839,
  4.430992282208889,
  4.347434072989289),
 71: (1.0010137638710939,
  0.05336056386113553,
  0.03689130248075371,
  1.001072749055473,
  4.6719663149893105,
  4.356481292663305),
 72: (1.0010035331560332,
  0.05152144271729305,
  0.03407454202526067,
  1.001069535774639,
  4.675948704401151,
  4.318943572622041),
 73: (1.0010133467070166,
  0.05025409278757773,
  0.0355274441623072,
  1.0010991581691187,
  4.706984724922819,
  4.469211006294374),
 74: (1.00103872421304,
  0.04799225072152847,
  0.034678317190555315,
  1.00111106699047,
  4.747261791561518,
  4.522030002434045),
 75: (1.0010537945165752,
  0.048916122961739104,
  0.034562390622999435,
  1.0011399296717824,
  4.893172777587766,
  4.678249007363964),
 76: (1.0010708116503542,
  0.04966257392911915,
  0.03151892711151405,
  1.0011406407015644,
  4.9990139581467945,
  4.606774486089828),
 77: (1.0011074021693485,
  0.05330094656215735,
  0.02955549065724989,
  1.001148211132933,
  5.1688300125642215,
  4.591695110628778),
 78: (1.0011180403178352,
  0.049112656066359686,
  0.028874066097180143,
  1.0011697779064674,
  5.096506430676857,
  4.655314137512003),
 79: (1.0011304572059416,
  0.04841150240200084,
  0.026947412173529892,
  1.0011677592034094,
  5.152398101773922,
  4.635063604691433),
 80: (1.0011534166529035,
  0.04553353872399892,
  0.024700924969237318,
  1.0011741027229923,
  5.167592764506428,
  4.64973278918232),
 81: (1.0011874751177516,
  0.04608734927115702,
  0.02305233470684396,
  1.0011910423241892,
  5.316733019992379,
  4.7044779053144365),
 82: (1.0012216530417395,
  0.05103072973892108,
  0.024090469721467257,
  1.0012143207041417,
  5.574171733626988,
  4.8419677252459685),
 83: (1.0012155910218057,
  0.04797959540599341,
  0.02060019659317827,
  1.001201637958086,
  5.571806837673038,
  4.788946500999864),
 84: (1.0012275595939713,
  0.04910439538311269,
  0.019857960969160853,
  1.0012032163768907,
  5.640324279276128,
  4.8032881191787995),
 85: (1.0012314523542247,
  0.05002031989635359,
  0.016917352746062733,
  1.001191677914485,
  5.663054741317092,
  4.696474927128238),
 86: (1.0012231554137863,
  0.04593165533890012,
  0.01721957326465501,
  1.001202043874071,
  5.567941410443215,
  4.7628173211612035),
 87: (1.0012255448411016,
  0.04368101466938563,
  0.019816155080097805,
  1.0012343996433413,
  5.527174476496044,
  4.925926300210491),
 88: (1.0012438685165126,
  0.045362415177245534,
  0.02081992458887493,
  1.001243630091461,
  5.640887063153161,
  5.0084158164963455),
 89: (1.0012529681222695,
  0.045004457443785825,
  0.018259690375348612,
  1.0012471442579247,
  5.673998014080022,
  4.943213170012531),
 90: (1.0012466479207036,
  0.043014067134372,
  0.020620794650688574,
  1.0012772145535302,
  5.64032722398762,
  5.096738203131195),
 91: (1.0012702065173769,
  0.03919037399383159,
  0.01884821331168683,
  1.0013013012804877,
  5.65402825580481,
  5.1466961576041985),
 92: (1.0013122519844961,
  0.04558703330429122,
  0.01942914361498943,
  1.0013390244055498,
  5.979987456808418,
  5.280096943144674),
 93: (1.0013238518956937,
  0.04726847057600329,
  0.017461316365187184,
  1.0013366753312876,
  6.059948204069907,
  5.243347406334702),
 94: (1.0013432211440945,
  0.05017160839262486,
  0.017760160273959286,
  1.0013385900967755,
  6.227867299852727,
  5.3368271920700625),
 95: (1.00138194743515,
  0.05338162142606559,
  0.01963600289835909,
  1.0013632592399198,
  6.458881003170041,
  5.516433884078345),
 96: (1.001381582879274,
  0.04946913740555234,
  0.01930037999772608,
  1.0013737444800777,
  6.390238529153018,
  5.574935374902541),
 97: (1.0013613147845113,
  0.04203645809025564,
  0.017203201204209467,
  1.0013796450507715,
  6.2625129774660895,
  5.597725239436265),
 98: (1.0013623974055619,
  0.03976736948692075,
  0.019415999912032945,
  1.0014089134131836,
  6.249321082440557,
  5.76285411621647),
 99: (1.00139726526146,
  0.04412004516156004,
  0.025739422885202843,
  1.0014562562514249,
  6.487622112286876,
  6.101108327904227),
 100: (1.0014010328638006,
  0.03796697427786529,
  0.024572631664173894,
  1.0014628283539142,
  6.383539464934106,
  6.1508963526377585),
 101: (1.0014149262114136,
  0.04137843578183782,
  0.02697268856463496,
  1.0014742044756928,
  6.490602438109776,
  6.262229276355189),
 102: (1.0014482705832934,
  0.046451183966788816,
  0.027184144090758873,
  1.0014865399033572,
  6.738645539197727,
  6.332576271079049),
 103: (1.001441702134815,
  0.03742094561902584,
  0.023041360279085977,
  1.0014959164165806,
  6.6149780715633595,
  6.345802076926896),
 104: (1.001470899238623,
  0.0399666230597874,
  0.022611364691556585,
  1.0015023160362937,
  6.8380501312686786,
  6.4108865977160665),
 105: (1.0014867384509618,
  0.04013115597118113,
  0.022118484692571144,
  1.0015260855565642,
  6.923321358271601,
  6.507334867695084),
 106: (1.0014829435376351,
  0.03459599900567687,
  0.01920173533916275,
  1.0015291577499232,
  6.88402741549989,
  6.541937768955268),
 107: (1.001489794752776,
  0.030185965272466513,
  0.017791319153445347,
  1.0015427911719266,
  6.8517261459552605,
  6.614788363863224),
 108: (1.0014715489793686,
  0.02263319320385256,
  0.019015449479443332,
  1.001570165378783,
  6.620603357920787,
  6.740712292523071),
 109: (1.001500162701251,
  0.02648792453317379,
  0.019990747350695923,
  1.0015989212740382,
  6.856752942813843,
  6.867842765389756),
 110: (1.0015509438817778,
  0.02724227809253148,
  0.02036227970026792,
  1.0016415012953963,
  7.0279327052132565,
  7.005081363564965),
 111: (1.0015668815656489,
  0.029448817637910834,
  0.023275567940757122,
  1.0016699408100305,
  7.166489885024688,
  7.1832409966368695),
 112: (1.001574210620731,
  0.028238452676094615,
  0.023834445539348378,
  1.0016775695764397,
  7.219612401679979,
  7.277345275352261),
 113: (1.0015797695672788,
  0.027537454115571673,
  0.024680652071500424,
  1.001706752910666,
  7.2958870917777014,
  7.431088809402564),
 114: (1.0015876297889528,
  0.02134987340602304,
  0.028117663094964714,
  1.0017499399414185,
  7.238044790518789,
  7.7060902304726255),
 115: (1.001637058151661,
  0.02677871108834392,
  0.018953030939650683,
  1.0017309729368442,
  7.644316425332043,
  7.464771623412588),
 116: (1.0016818510558196,
  0.03236412920291438,
  0.016941239211921012,
  1.0017314468890584,
  7.940030314212482,
  7.438644962181209),
 117: (1.0017071445778736,
  0.02628104344114239,
  0.013359352125705696,
  1.001750707808061,
  7.891921812119133,
  7.433340341929358),
 118: (1.00175125155169,
  0.028049322362313465,
  0.011754147243330194,
  1.0017767127016353,
  8.109698590771941,
  7.525562714389231),
 119: (1.0017691132097544,
  0.025328851731911364,
  0.015227420589668935,
  1.0018401096480538,
  8.138600840067866,
  7.831474719459001),
 120: (1.0017546789884808,
  0.018737382269375973,
  0.008593788821656113,
  1.0018283305349545,
  8.038537145215948,
  7.670300782551975),
 121: (1.001769085060014,
  0.017198352238636333,
  0.007490075682761598,
  1.0018508239437725,
  8.071467206134749,
  7.728350718696934),
 122: (1.00180308725956,
  0.017078727318170065,
  0.002635814446259691,
  1.001852102837873,
  8.17141204870989,
  7.631328646149375),
 123: (1.0018450596501052,
  0.025174423895821675,
  0.0038034163839428424,
  1.0018802855336497,
  8.508271553555526,
  7.730114313851443),
 124: (1.001868716780902,
  0.0278699451126172,
  0.006898295442192792,
  1.0019098292467803,
  8.674463968869048,
  7.961878222634891),
 125: (1.0018975943961956,
  0.025558639424699187,
  0.00720302091140843,
  1.0019416902575486,
  8.719008720267835,
  8.086630186820486),
 126: (1.0019321075344925,
  0.030257016026100612,
  0.009779085168633909,
  1.0019896055669617,
  8.980513235319998,
  8.302325562149454),
 127: (1.0019434802840155,
  0.026419311346363408,
  0.006890119948983493,
  1.0020049643264584,
  9.013813156726519,
  8.33163103613314),
 128: (1.0019649443283094,
  0.024819346968139352,
  0.008700023842445224,
  1.0020214993877548,
  9.052787545162387,
  8.46883571406108),
 129: (1.001977466045828,
  0.025602368331860637,
  0.007656998926756696,
  1.0020228684827135,
  9.264991814102146,
  8.570812050566106),
 130: (1.0019880400527548,
  0.023667548334921337,
  0.004921442060685271,
  1.0020199403676828,
  9.227609914267601,
  8.49210747315819),
 131: (1.002012808656276,
  0.02306758178301367,
  0.001314709060229052,
  1.0020134557439981,
  9.281250255733406,
  8.395547033640433),
 132: (1.0020142931848017,
  0.01693780417362251,
  0.0027107467356588012,
  1.0020422575863144,
  9.193813887839951,
  8.607520248931708),
 133: (1.0020566752649949,
  0.02470093527526524,
  0.005317140009844705,
  1.0020894229051642,
  9.508031083347621,
  8.808864489497948),
 134: (1.0020671798303653,
  0.02278021404054301,
  0.004839560672355792,
  1.002108157932218,
  9.580582232353558,
  8.913717226204191),
 135: (1.0020545961823053,
  0.01083188910692524,
  0.0007168892147971992,
  1.0021025981677067,
  9.310006256558987,
  8.88373125377084),
 136: (1.0020819029166785,
  0.012982215474751509,
  0.003260515465017936,
  1.0021524524449537,
  9.490352307116238,
  9.097447278056787),
 137: (1.002109180964236,
  0.015690414259106143,
  0.0020357953648629003,
  1.0021686192761907,
  9.653761410090537,
  9.116800533432908),
 138: (1.0021347563864977,
  0.01277503271677691,
  0.0022416341221032344,
  1.0022122853506348,
  9.706648627847358,
  9.301233114588094),
 139: (1.0021467645240343,
  0.01239283743621908,
  0.004974770308221684,
  1.002254247595866,
  9.808506194749775,
  9.517054086232005),
 140: (1.0021531263959906,
  0.01145431909264124,
  0.005968666978669057,
  1.0022691603170968,
  9.841185810692998,
  9.615961196728735),
 141: (1.0021636129379816,
  0.012860474410313532,
  0.007166794773867083,
  1.0022844451757713,
  9.9334776367762,
  9.736766019305794),
 142: (1.0021717067834552,
  0.0077128345348900975,
  0.0050531599413367705,
  1.0022823628079773,
  9.858195652628702,
  9.724085749556068),
 143: (1.0021957169862827,
  0.008150060814023351,
  0.0069641093808485675,
  1.0023104271672316,
  9.937004586752604,
  9.88634644292063),
 144: (1.0022227436436295,
  0.009124131247707386,
  0.004897834540664508,
  1.0023198603308,
  10.082870304596781,
  9.88209853129006),
 145: (1.002244660234744,
  0.010330327697804727,
  0.004619950757925355,
  1.0023376700099085,
  10.211732957604355,
  9.938269227639648),
 146: (1.0022509357955849,
  0.004546272406170797,
  0.0032070628517457474,
  1.0023499522238073,
  10.109823367661134,
  10.007119784592302),
 147: (1.0022897146001202,
  0.007009077859381392,
  0.0019056902881325667,
  1.0023603969312305,
  10.249674556962448,
  10.004899338281733),
 148: (1.0023307339375538,
  0.011854440472387439,
  0.001395537500113317,
  1.0023836674565736,
  10.562623227085258,
  10.091395779709021),
 149: (1.0023491080827485,
  0.010023367509582852,
  0.0016006347717150934,
  1.002405795274947,
  10.569876775313931,
  10.17031646249389),
 150: (1.0023415302697172,
  0.005033589388035424,
  0.0030580317013123304,
  1.002433510493446,
  10.463345903076918,
  10.288912199920809),
 151: (1.002355187464412,
  0.00743856262409494,
  0.0014666261671097273,
  1.002449103175267,
  10.578008328717418,
  10.290969381299027),
 152: (1.0023779639797088,
  0.006611598905641183,
  -0.0009694616343345508,
  1.0024603191385222,
  10.647795929263976,
  10.26815294721426),
 153: (1.0023790503810388,
  -0.000982009381752364,
  -0.003327855850753675,
  1.002487805648212,
  10.501452276263736,
  10.317654090534337),
 154: (1.0023897385072735,
  0.0005008292798210367,
  -0.00445100830207857,
  1.0024859716108288,
  10.634952019200378,
  10.335173135623027),
 155: (1.0024362890140468,
  0.004387502551718858,
  -0.0035224457675954624,
  1.0025140734756244,
  10.895892979562955,
  10.447644622501503),
 156: (1.0024466941909564,
  0.004699089211273156,
  -0.0003068668122848963,
  1.002561952379878,
  10.967113915372748,
  10.68301361164105),
 157: (1.0024554494000404,
  0.003423055595332399,
  -0.0014416670133380423,
  1.0025735090184824,
  11.011909110739653,
  10.732587967512329),
 158: (1.0024954148673397,
  0.01123463650019501,
  -0.001811072626681765,
  1.002590733166248,
  11.402080297169928,
  10.841766000972447),
 159: (1.0025051015835722,
  0.008742002033327622,
  -0.00401983393702075,
  1.0025977394525125,
  11.406249926717946,
  10.82369169020444),
 160: (1.0025570245947704,
  0.014855113112736823,
  -0.006992764462789476,
  1.0026105810429808,
  11.728669225766044,
  10.794607122723153),
 161: (1.0025359563440432,
  0.009325001855811949,
  -0.006238214456332657,
  1.002630774106633,
  11.553168385270249,
  10.872059976477775),
 162: (1.0025637628994528,
  0.009866145652299872,
  -0.00563424363817018,
  1.0026501411813078,
  11.68133323362323,
  10.984178282247512),
 163: (1.0025683451519263,
  0.00820515215993003,
  -0.006566921600087148,
  1.0026515364772985,
  11.645487917927877,
  10.976362115703152),
 164: (1.0025708974079064,
  0.0036928939194679506,
  -0.005910005869683132,
  1.0026852235294212,
  11.549738995436348,
  11.130181727987274),
 165: (1.0025661456923456,
  -0.001947337797611505,
  -0.0038312347685751135,
  1.002720535989897,
  11.458101002160642,
  11.331800447991816),
 166: (1.0026045020969392,
  0.0014724612997977851,
  -0.002981649886954401,
  1.0027596099262128,
  11.67156762422606,
  11.447245699821938),
 167: (1.0026179188511732,
  -0.0009988831204435712,
  -0.007349765288631109,
  1.0027608479314447,
  11.767708714535905,
  11.397670115012213),
 168: (1.0026328177169597,
  -0.0012009088006586367,
  -0.0053337524404110945,
  1.0028011026565333,
  11.837356199495852,
  11.548922382102818),
 169: (1.0026558099372855,
  0.0016127174005692597,
  -0.004173752289507804,
  1.0028209155926329,
  12.034539276742832,
  11.665876722964214),
 170: (1.0026812322471446,
  0.004288232446802556,
  -0.007083763040147406,
  1.0028234422345583,
  12.189257114560855,
  11.625397078585232),
 171: (1.0026727108133495,
  4.151450614579225e-05,
  -0.006304579706448392,
  1.0028379936273555,
  12.100920153893798,
  11.71120228951842),
 172: (1.0026918374815659,
  -0.00025045649166148456,
  -0.010624145002741602,
  1.0028441568903979,
  12.206653464299778,
  11.68009612136636),
 173: (1.0027087934437167,
  -0.000616435725898457,
  -0.011674723356247174,
  1.002850980752777,
  12.274434388035639,
  11.687624939643866),
 174: (1.0027027995977735,
  -0.004062567399148743,
  -0.013456790525997589,
  1.0028523991610334,
  12.184938059885772,
  11.668671820259435),
 175: (1.002717948252488,
  -0.0019180976676499155,
  -0.012127969858136512,
  1.0028759159699396,
  12.271589527895543,
  11.755007397104539),
 176: (1.002768517000085,
  0.005535949234591558,
  -0.013487806812915394,
  1.0028906393693426,
  12.608494713594945,
  11.771240908075693),
 177: (1.0027568328451135,
  -0.003916800322814317,
  -0.011560820482554506,
  1.0029285915774904,
  12.357598239126132,
  11.953544412097118),
 178: (1.002813683361649,
  0.001961950109245628,
  -0.013304825374824159,
  1.0029393180800068,
  12.700242065509965,
  11.956347569140414),
 179: (1.00279789771569,
  -0.003193374892262784,
  -0.01288022034763258,
  1.0029579821002805,
  12.559326146592907,
  12.025573281023272),
 180: (1.002802586201974,
  -0.004409984121811695,
  -0.012365375707334543,
  1.0030022965984196,
  12.576081151667736,
  12.153860219616265),
 181: (1.0028171010961264,
  -0.0041686383256104974,
  -0.013557369938517947,
  1.003008819069466,
  12.670044116682961,
  12.172200779550856),
 182: (1.0028336261164101,
  -0.004659945960625248,
  -0.013078205409303186,
  1.0030229419991026,
  12.73149711062484,
  12.261402751806095),
 183: (1.0028604713353673,
  -0.0015658607855911994,
  -0.012098971139120108,
  1.0030481953989219,
  12.886170487293693,
  12.348160722182765),
 184: (1.0028391199434779,
  -0.0060682743974767965,
  -0.012710025423094403,
  1.003051980482818,
  12.713402110572996,
  12.35446930484515),
 185: (1.0028716066519192,
  -0.0022402529215430306,
  -0.010531800997446663,
  1.003065705525438,
  12.908507468934959,
  12.449980443294773),
 186: (1.002835145455527,
  -0.012918649903239427,
  -0.01177820964488092,
  1.0030741856208256,
  12.58946128936414,
  12.458567659637572),
 187: (1.0028520671829444,
  -0.007926316106780445,
  -0.013247294121776625,
  1.0030846276708403,
  12.742940663835764,
  12.399122148782187),
 188: (1.0028780520908482,
  -0.0034610068081387738,
  -0.014222028826517373,
  1.0030923876467457,
  12.886803620486722,
  12.373312430926967),
 189: (1.002902656929669,
  0.00020577747701793197,
  -0.01420267603645219,
  1.0031073723328114,
  13.06221091709228,
  12.446303575932541),
 190: (1.0028867480131376,
  -0.005844044675562019,
  -0.01387046843890957,
  1.0031357310484912,
  12.882414795728227,
  12.566537430563868),
 191: (1.0028958526262244,
  -0.007568305207781051,
  -0.010605578847616644,
  1.0031755396313145,
  12.852062833325622,
  12.783826271634977),
 192: (1.0028963479903135,
  -0.009815130463589216,
  -0.009754698885980372,
  1.0031843810024403,
  12.802034975401911,
  12.857856947593103),
 193: (1.0029023334878442,
  -0.009283392377352316,
  -0.010583307768849946,
  1.0031926652781853,
  12.814046730648196,
  12.846984324494825),
 194: (1.0029502435355997,
  -0.001769953603955864,
  -0.012804604515768075,
  1.0031941136052003,
  13.112260351920057,
  12.77927424946506),
 195: (1.0029537393766395,
  -0.0034345249773231864,
  -0.013593367756741328,
  1.0031937693307251,
  13.082299346551224,
  12.727718597511581),
 196: (1.0029523340382582,
  -0.0065680165445112775,
  -0.011275040609366141,
  1.0032218753367068,
  13.016930425149353,
  12.874030691163242),
 197: (1.0029877604510269,
  -0.0027191514271736445,
  -0.010622643176869567,
  1.0032301481911745,
  13.232374787855141,
  12.919664502155081),
 198: (1.0029982208042074,
  -0.004891241855731582,
  -0.00749662780329246,
  1.003262644468391,
  13.189267879903944,
  13.085748816506415),
 199: (1.0029896781839711,
  -0.008073903291134716,
  -0.01401479995489549,
  1.0032479288751739,
  13.162486728355946,
  12.874730622712242),
 200: (1.0029839517261592,
  -0.010245644582257227,
  -0.016416327002220565,
  1.0032415646265085,
  13.106555670896517,
  12.761316915069711)}

x_vals = [descent_data[i][X_INDEX] for i in range(0, n_iterations)]
y_vals = [descent_data[i][Y_INDEX] for i in range(0, n_iterations)]

We see in the plot that the metric generally improves as transformation parameters are updated with each iteration, but the final position may not align with the maximum position on the plot. This is one case in which it is difficult to visualize gradient ascent over a hyperdimensional space, where the optimizer is stepping through six parameter dimensions but the 2D plot we collected with ExhaustiveOptimizer represents a ‘slice’ in space with x[0:4] fixed at (1,0,0,1). Here it may be more useful to directly compare the two images after registration to evaluate fitness.

fig = plt.figure()
# Note: We invert the y-axis to represent the image coordinate system
plt.gca().invert_yaxis()
ax = plt.gca()

surf = ax.scatter(X, Y, c=Z, cmap=cm.coolwarm)

for i in range(0, n_iterations - 1):
    plt.plot(x_vals[i : i + 2], y_vals[i : i + 2], "wx-")
plt.plot(descent_data[0][X_INDEX], descent_data[0][Y_INDEX], "bo")
plt.plot(descent_data[n_iterations - 1][X_INDEX], descent_data[n_iterations - 1][Y_INDEX], "ro")

plt.plot(max_position[X_INDEX], max_position[Y_INDEX], "k^")
plt.plot(min_position[X_INDEX], min_position[Y_INDEX], "kv")

[<matplotlib.lines.Line2D at 0x7f824a761070>]

Resample the moving image

In order to apply the results of gradient ascent we must resample the moving image into the domain of the fixed image. The TranslationTransform whose parameters have been selected through gradient ascent is used to dictate how the moving image is sampled from the fixed image domain. We can compare the two images with itkwidgets to verify that registration is successful.

ResampleFilterType = itk.ResampleImageFilter[ImageType, ImageType]
resample = ResampleFilterType.New(
    Transform=transform,
    Input=moving_image,
    Size=fixed_image.GetLargestPossibleRegion().GetSize(),
    OutputOrigin=fixed_image.GetOrigin(),
    OutputSpacing=fixed_image.GetSpacing(),
    OutputDirection=fixed_image.GetDirection(),
    DefaultPixelValue=100,
)

resample.Update()

view(resample.GetOutput())

<itkwidgets.viewer.Viewer at 0x7f824a6edfa0>

The image comparison shows that the images were successfully translated to overlap, but were not fully rotated to exactly align. If we were to explore further we could use a different optimizer with the metric, such as the LBFGSBOptimizer class, which may be more successful in optimizing over a rough parametric surface. We can also explore different metrics such as the MattesMutualInformationImageToImageMetricv4 class to take advantage of the ITK v4+ registration framework, in contrast with the MutualInformationImageToImageMetric used in this example as part of the v3 framework.

Clean up

os.remove(fixed_image_path)
os.remove(moving_image_path)