Seam Carving with Julia

I stumbled upon this concept of a content aware image resizing algorithm and thought it seemed really cool. I found it because Grand Sanderson (3blue1brown) was teaching it in a free course for MIT called Computational Thinking. I like his teaching so I had to check it out.

Julia is a relatively new programming language. It promises high performance much like C but comes with nicer syntax and packages like Python or MATLAB.

The whole program is written in a notebook program called Pluto.
It is easiest to view the code as a html document here!

Julia was a very nice language to learn. I just touched the surface but having some past projects in MATLAB it felt a bit familiar.

The final product turned out pretty well for certain images like paintings.

In the demo below you can see how the algorithm tries really hard to avoid removing parts of the image with detail. The fox and forest does not get deformed at all while the sky and hill shrinks.

julia sobel filtered.png

The first step was to process the image with the find_energy function.
I used the Sobel operator which when multiplied over each pixel in both x and y directions produces high values where edges are.
Combining all these values back into an image we get the result above.

julia energy map.png

Step two involves figuring out the energy_map. Because a seam needs to be connected, we cannot just remove the lowest value from each column, we have to make a map where each value corresponds to the least energy that would have to be removed if this pixel was to be removed.
A second map with information of the path for each value is also returned.

julia seamed image.png

Finally the function find_seam uses all the data produced to find the column or seam with the lowest accumulated energy from bottom to top.
Here I painted the first seam in red. It avoids all white portions from the energy_image.

In the function get_all_carved I call find_seam and remove_seam recursively on the image produced by the previous call.

The notebook in julia format can be found at my GitHub!