Image Scaling: Introduction to Scaling, Resizing & Resampling

Once an image has been rendered and rasterized, the scale may no longer be able to widen (or scale up) depending on its number of pixels. If the pixel count has decreased (or scaled down), usually the quality of the image will also decrease.

A good way to understand this effect is to imagine that all image scaling and resizing takes place on a grid — the finer the grid, the more detailed the results. However, before rendering even takes place, that pixel data often needs to be resampled or interpolated. There is no single correct approach for data interpolation, and results can differ depending on the interpolation techniques you use. These techniques may come across as complicated and downright incomprehensible to the layman, so in this article, we hope to make things a little easier by explaining some of the most popular techniques that can help you with processing your next project.

Nearest-neighbor Interpolation

The most common and basic approach to expanding image sizes is called nearest-neighbor interpolation (or round interpolation), which calculates the average or closest value of each pixel and replaces it with the closest matching pixel and intensity value, resampling into the render’s output. This potentially results in multiple pixels of the same color showing up in the final rendering.

Pixel art regularly uses this technique as it’s the fastest interpolation method, requires the least amount of processing time, and allows for sharper detail. The biggest drawback, however, is that the resulting image may contain jagged edges.

While you may assume that “nearest” refers to the nearest mathematically, that is not necessarily the case. A common approach is to round towards zero because it creates less artifacts.

Bilinear and Bicubic Interpolation

As mentioned earlier, it’s best to observe image scaling and processing as if it were set by a grid (or a matrix, for the technical minded). Linear interpolation uses this grid most succinctly; but seeing as how for images these interpolations are happening in two dimensions, this method is usually referred to as bilinear. Bilinear interpolation takes into consideration the closest 2×2 neighborhood of known pixel values surrounding a pixel. It then calculates a weighted average of those pixels to come up the final interpolated value. This results in an image with a smoother appearance compared to nearest-neighbor interpolation.

In contrast to this type of interpolation, taking only four pixels (2×2 pixel values) into account, bicubic interpolation takes 16 pixels (4×4 pixel values) into account, making for an even smoother albeit slightly blurry result. Rendered animations resampled using this interpolation algorithm are also smoother and have fewer interpolation artifacts. With still images, however, the differences are very minimal and not easily noticeable.

When processing time is not an immediate issue, bicubic interpolation is more often chosen over bilinear or nearest-neighbor interpolation during processing.

Box Sampling

One downside to bilinear or bicubic interpolation is that the sampled pixel count is specific, mainly in grids of 2×2 or 4×4. There comes a scaling threshold that results in lost data and fidelity when the algorithms include non-adjacent pixels in the sample, leading to imperfect results with artifacting and distortion.

Box sampling can overcome this issue. This method takes the target pixel of a box on the original image, and samples every pixel within the box, ensuring that all input pixels contribute to the output. While effective, this method can be difficult to optimize.

Another important drawback of such an algorithm is the amount of resources used when compared to other techniques that are simpler and need less resources. The common thing among all three options is that after certain input latency, the pixel value is calculated at the data rate. Area averaging or a box filter, on the other hand, is simple, fast, and near optimal.

Sinc (Lanczos) Interpolation

The last algorithm we will mention is also probably the most complicated and hardest to pin down in terms of its usage. The Lanczos interpolation method provides the best properties when it comes to preserving detail, and minimizing aliasing and artifacting. The process in which it accomplishes this is much more complex than any of the other methods discussed earlier. With the input samples of an initial image, they are then filtered through the Lanczos kernel (or process) to reconstruct them. Once this process is completed, the interpolation is implemented using a mathematical convolution function known as the sinc function. (Like we said, it’s complicated.)

This method seems to have an interpolation result that’s similar to the bicubic interpolation method. However, the algorithm tends to have more of a ringed effect rather than a blurry effect. The effect sometimes will have a dark or light halo along any sharper edges of the rendered image. Though the effect may not be desired, it increases the perceived sharpness of the image. The takeaway is that the Lanczos method can offer a perceived cleaner, sharper image rather than a blurry one. It’s definitely a more preferred method in video upscaling than with still images.

The Best Image Scaling Method Requires Testing

Be sure to keep in mind that no single algorithm is superior over another. Image scaling has everything to do with what you need and how you judge the quality of the output. Some designers prefer nearest-neighbor interpolation for game rendering and others prefer Lanczos interpolation for graphic design and text. What one artist relies on will not necessarily be what another artist needs. If your software has the capability of testing these options with a resize editor or compositor, we recommend testing all of the options available to you to see which process is best at making your project look visually stunning. Happy rendering!