Reconstructing photos

This is an experiment on reconstructing a picture using the chunks of another picture. That is, take one picture (the brush picture), divide it into a reasonable amount of rectangular “chunks”, and then try to construct another picture (the target picture) by reordering the “chunks” of the first brush picture. Sounds trivial, but the real art is in how to do the reordering.

For those mathematically oriented, this is a problem of finding the permutation of the array of “chunks” which minimizes the difference between the permutation and the target image according to some metric.

The main picture above was constructed by using two pictures below,

img1 img2

First the brush image was divided into 64*64 chunks of equal size and a value of the metric was calculated for each of the chunks. The metric was here defined by the average RGB-value of the pixels in the chunk and the normalized variance of each of the color channels of the pixels in the chunk, hence the overall metric was actually a six-dimensional vector which characterizes the color and color variation of the chunk. Then the target image was split into equal number of chunks and the same metric was calculated for each of the target-image chunks. After that, it was a simple matter of finding the nearest chunks of the brush and target images in terms of the given metric.

Watch a video of the reconstruction in action:

I experimented with different definitions of the metric to match the chunks (brightness-based for grayscale images, pure RGB or color variance, color histogram), but in terms of quality and effectiveness the forementioned 6D-metric was the most productive.

This is really just the process of video compression applied for artistic purposes. Still there are many interesting development ideas for the interested reader. It would be fun to find the smallest number of non-trivial base chunks which would be reasonably able to reconstruct some set of images, or to reconstruct a movie using only different permutations of the chunks of a single still image.

The image segmentation and matching was done in Processing using a sketch I wrote. I decided not to publish the applet, since it would not provide anything usable for the public.

Published by


Flâneuring through experiments in data, science and artsy digital things.