Smooth Operations: Anti-Aliasing, Interpolation & Dithering

When it comes to creating illustrated imagery, one type of element that is important in every illustration, and is often taken for granted is an edge. In the non-digital illustration world edges are created by the physical use of an art material, but in the digital world, edges must be created in a virtual way to simulate a non-digital visual appearance. Image software programs like Adobe Photoshop provide both automated and user defined ways to manipulate or smooth an edge. So, as is the case with every aspect of a digital image, the illustrator has the ability to influence the look of edges in their work. It’s helpful to know how digital edges are rendered and displayed in order to do so.

Anti-Aliasing

A good explanation of anti-aliasing begins with a good description of display aliasing. Basically, aliasing is a jagged or stair-step appearance that is caused by a lack of resolution either by restrictions in a hardware device, such as a display, or file size limitations in a raster-based document. Anti-aliasing is the technique software designers have adapted to reduce or eliminate jagged edges by fooling the eye of the viewer. This is done by creating a diminishing series of shading steps between the edges of adjacent colors. See Below. Anti-aliasing is also referred to as “smoothing”.

Closeup of aliased stair step in red, anti-aliased smoothing in blue.

Aliased mountain rendering. © 2012 Don Arday.

Anti-aliased mountain rendering. © 2012 Don Arday.

Closeup of the anti-aliased edge of a rose petal.

Interpolation

Anti-aliasing occurs through the interpolation of picture elements, “pixels”. Think of interpolation as swapping and sharing. To improve the digital visual appearance of an image on our monitor the technique of interpolation is used. Let’s refer to the data in an image as a sample. To make the image display better, the computer converts the image to a higher sample by using filtering techniques like anti-aliasing. It may be no coincidence that the word interpolation bears a resemblance to interpretation, because that is sort of what occurs within anti-aliasing. The software in the case of a document, or the computer video card in the case of the display, interprets the color relationships when attempting to create a smooth transition between one color and another to render edge pixels. This generally occurs automatically, but illustrators can use manual methods to control the appearance of edges within their illustrations.

Dithering

Dithering is often confused with anti-aliasing, but it is actually something quite different. Dithering most often occurs in color image situations where colors in an image are out of the range of the colors available in an image display or output device. When this occurs the software program creates the unavailable color by mixing other colors that are available. For example, dithering can occur when a particular web browser does not support a specified color. The browser will then attempt to mix the requested color by “dithering” pixels by creating a combination from other colors it has available. Although usually invisible to the naked eye, dithering is done in the same manner as “noise” is, (see below) and like noise, it can appear slightly grainy in the extreme.

Some Manual Techniques

Feathering

Feathering is the softening of edges of adjacent shapes and colors within an image beyond the standard level of interpolation. Adobe Photoshop gives you the option to feathering selection edges in an image. A feathered edge results in increased edge blurring and a lowering of edge contrast between colors. With feathering, the artist has the ability to control the number of pixels to be included in a feathered edge. The result of feathering is similar to that of Gaussian blur. (See below.)

Gaussian Blur

Using Gaussian blur, or any of the other blur options in raster programs, means to force adjacent colors to drift into one and other creating a smoother edge transition. Colors in a Gaussian blur drift in a randomly distributed, non-linear manner. A motion blur option of the Gaussian blur limits the random color distribution to a particular direction. Blur can be used to redefine edges, and it can also be used to control the amount of focus in an image.

Gaussian blur applied to complimentary colors.

Gaussian blur closeup.

Noise

Noise is an exaggerated form of dithering. Although a controllable option in Photoshop, and a very useful one, noise, and specifically random pixel noise, is not necessarily thought of a s a good thing. Pixel noise is an unnatural, or unwanted, variation of value or color information in an image. Noise has a tendency to degrade sharp edges within an image by producing a scattered effect of random colored pixels. So a yellow flower against a blue sky may end up with blue pixels mixed among the yellow ones. And the flower and the blue sky might both have red pixels scattered in as well. See below. Photographers generally find noise to be very disturbing, however for the illustrator it may help to produce a desired, even naturalistic color effect. Noise is best applied as a final stage before publishing an illustration. And once saved an illustration that has had noise added is permanently altered, so it is always best to save your illustration prior to using noise, then save the noise applied version as a separate file.

Noise applied. Note the addition of other color pixels to the original pair 
of complimentary colors.

Noise closeup.

Sharpening

The three techniques discussed above can produce a smoothing appearance to an edge, but they also degrade its sharpness. Using a sharpening filter or “Unsharp Mask” in Photoshop can give the appearance of actually sharpening edges. Sharpen filters identify where dark and light edges meet in an image, and add contrast to those edges. The filter does this by creating an even darker line along the edge of a dark shape, and also a lighter line on the edge of an adjacent light shape. Unsharp Mask tends to move edges away from an anti-aliased appearance, and taken to far, sharpening can produce a very unnatural looking halo around shapes. See below. Sharpening like noise is best applied as a final stage before publishing an illustration.

Edge sharpening applied. Note the increased contrast at the edges of 
the two colors.

Edge sharpening closeup.

There are also some techniques in Photoshop for working with edges that rely on methods of selection like “Refine Edge” and “Fringe”, but the result of any of those actions uses the above basic methods of rendering.

By understanding how digital image edges are rendered and by using these manual options for refining them, you will have greater control over the appearance of your illustration either in print or on the monitor.

Vector, Raster, and Vexel Defined

Digital illustrators use programs that are based on raster or vector interpretation of data, and all of us who have worked with Adobe Photoshop and Adobe Illustrator have experienced how different the workings of the two programs can be. Illustrator tends to be used more by illustrators and graphic designers that tend to focus more on object making. Illustrator is referred to as “object oriented” software. Photoshop tends to be used more by illustrators, photographers and designers who lean more toward making pictures. Of course these assumptions are not absolute, and the developers of both Illustrator and Photoshop have blurred the lines between raster and vector as they have evolved. This comes from each program attempting to provide a totally functional software solution. For instance, Illustrator uses some raster display effects and offers the user an option to rasterize elements and to use raster painting effects. Alternatively, Photoshop offers Bezier pen tool functionality, the option to use vector masks, and to create and save outlines. So with the cross adaptation between raster and vector, these software programs offer illustrators new options that didn’t exist a few years ago. And on top of all this, software programs like InDesign allow the use of both raster and vector components in a single document. One thing that can help illustrators and graphic designers make the decision as to which direction to go is to consider the purpose for the illustration, i.e., the software choice is determined by how the image must function. In order to make that decision an understanding of properties of raster, vector, and vexel imaging is indispensable.

Vector

Vector software is object oriented, a collection of objects that always retain their integrity. Straight and curved lines, gradients, and shapes including letterforms are an expression of mathematical descriptions. The positions, scale, and display attributes of all objects are noted mathematically using algebraic equations. Altering objects within an image results in an alteration of the mathematical data. This form of image processing preserves the integrity of the objects themselves. Because of this, there is no image degradation as a result of changing any of the objects attributes.  Even when an object, say a perfect circle, cannot be viewed properly on a monitor without some distortion, because monitors display in pixels (picture elements), the circle will exist as a perfect mathematical construct. And, with the right output device will appear so.

Illustrator vector file viewed in outline mode. © 2000 Don Arday.

Vector graphics are also referred to as “device resolution independent”, which is sort of an oxymoron. Although a vector document is dependent on a monitor or a printing device in order to be seen, the data itself always remains independent of any form of display or output. For instance, a vector image printed on a high-resolution 9600dpi printer will have a resolution of 9600dpi, and the same file printed on a low-resolution 300dpi printer, will have 300 as its resolution, and so on, but the original document data does not change.

Illustrator vector file viewed in preview mode. © 2000 Don Arday.

Vector files are much smaller than raster ones due to vector documents being based only on mathematical descriptions and not on pixels. For this reason vector files are extremely portable and very well suited to certain functions. Resizing a vector image is done by multiplying the mathematical description of the objects in the image by a scaling factor, so a file size of a vector illustrations will be the same whether it is output as a 3” x 4” spot illustration or it has been resized to be output as a 12’ x 16’ mural.

Strengths

Small, efficient file sizes.

Infinite scalability of images without a loss of quality.

Excellent type rendering, manipulating, and editing abilities.

Ability to apply mathematical operations to image components.

Accuracy in rendering geometric forms.

Ability to edit image elements.

Device resolution independent.

Weaknesses

Ability to render continuous tone full color images.

Ability to globally edit color brightness, contrast, hue, saturation and value.

Ability to apply editing filters to images.

Raster

Raster software is pixel oriented. Raster files are made up of individual picture elements, or pixels that are perfectly aligned arrays of color and value. Pixels run edge to edge, covering the entire surface of the image. Straight and curved lines, and shapes are all combinations of pixels. The size, position and attributes of parts of an image are all relative to their makeup in pixels.  The resolution or size of a raster illustration relates to, and is limited by, the number of pixels contained within it. As a side note, raster images display truly on monitors because both raster images and monitor displays are made of pixels, whereas monitors simulate vector images, which have no pixels.

Raster images are “device resolution dependent”. The output quality of an illustration is dependent upon the resolution required by the display or output device. This means that illustrators must know the intended function for their illustration in order to properly create it. For instance, a 3” x 4” Photoshop illustration for use on the web can be created at 72ppi (pixels per inch), and it would display smoothly, but the same file printed on a 1200dpi image setter would look jagged and “pixilated”. In other words, the pixels that make up the illustration would be conspicuous to the viewer making the image quality appear very poor. Conversely, that same 3” x 4” illustration created at 300ppi would print nicely, but used on the web, it would slow down or pause the loading of a webpage. Due to device resolution dependence images are not upwardly scalable. (See Digital Image Resolution a prior post for more information on scaling.)

Photoshop raster version enhanced by raster image operation editing. © 2000 Don Arday.

Raster files are much larger than vector files. This is because all the attributes, including the location and color of every single, individual pixel must be recorded, not to mention any saved operations or specialized layer data. All this can result in massive file sizes for raster illustrations; especially those that are created for high-resolution output devices.

Strengths

Ability to render continuous tone full color images.

Ability to globally edit color brightness, contrast, hue, saturation and value.

Ability to apply editing filters to images.

Weaknesses

Large file sizes.

Poor scalability of files.

Poor ability to render, manipulate, and output type.

Poor ability to generate geometric forms.

Device resolution dependent.

Vexel

Vexel art is where the style and method of rendering in vector based software programs is produced in a raster based environment. The term was coined by combining the words vector and pixel. As raster software programs like Adobe Photoshop have become more sophisticated they have incorporated vector based drawing tools such as bezier curve functions, shape generators, and text application functions. Therefore it is possible to get a vector art style appearance that is composed of pixels rather than objects. And it is possible to apply raster techniques to vector based components.

Strengths

Ability to render a vector style image in a raster environment.

Ability to use and combine both vector and raster operations in a single environment. 

Ability to render continuous tone full color images.

Ability to globally edit color brightness, contrast, hue, saturation and value.

Ability to apply editing filters to images.

Weaknesses

Large file sizes.

Poor scalability of files.

Limited vector style operations.

Device resolution dependent.

 

Cross Platform Digital Image File Formats

As illustrators we use a few digital file formats in the course of the work we produce, and even though knowing about them is important, we tend to take them for granted. Especially when the software we are using offers up convenient default options for formatting and saving our files. 

This post is going to highlight cross program image based file extensions such as .tiff, .eps,, etc.; rather than application native ones such as .AI (Adobe Illustrator), .PSD (Adobe Photoshop Document), or .procreate, etc. Also it is going to focus on those that are most commonly used by illustrators.

The Basics of File Extensions

Firstly, it is important to know what a file extension is. A file “name” extension is the suffix that is attached to a filename. For instance, My Illustration.tif, where “My Illustration” is the file name and “.tif” is the extension. The extension identifies the type of file I have saved. The important thing to note is that the file extension is the code that a software program will use to recognize and open the file. And in the case of the .tif extension, there are many software applications that can recognize and open a .tif file.

File extensions can be application specific, cross-program, or cross-platform.  Since each software application usually has its own file extension, there are tens of thousands of file extensions patented. Nowadays, the file extensions suffix is automatically attached to the file name. This is not only essential for a software program to recognize the file, but it is also very helpful for a user. Back in the day suffixes weren’t displayed, so one often had to go to “File Info” to check the format. In the too much information category, the longest file name extension registered at this time is the 25 letter + dot “.bejeweled2deluxesavedgame”. This doesn’t help when the total length a file name including the extension can be is limited to 255 characters.

The Reason for File Formats

In a nutshell, the purpose of a file format is to preserve the accuracy and appearance of a digital document for a desired context. That’s why it is very important to know the purpose, uses, and limitations that were designed into the various file formats to avoid usage mismatches or loss of data. For instance, saving a 24-bit RGB Adobe Photoshop document in a .gif format will reduce the number of colors from 16,777,216 down to 256, i.e., 24-bit down to 8-bit, which could have drastic consequences depending on the output requirements for the file.

The Formats

TIFF

TIFF stands for “Tagged Image File Format”.
.tif files can be raster or vector based images. .tif supports black-and-white, grayscale, index color (256 color), RGB, LAB, and CMYK images and offers excellent compression options including  LZW (Lempel-Ziv-Welch) lossless compression. A .tif file embeds an ICC color space profile, which means many applications can access .tif files. .tif files can be reopened and edited. A non-flattened Adobe Photoshop .tif file will save layers, adjustment layers, channels, paths, and transparencies, but programs that can read a .tif may not be able to read all of the these features. Presently, Photoshop supports 16, 24, and 32-bit depth images.

JPEG

JPEG stands for “Joint Photographic Experts Group”. The .jpg format operates strictly in the realm of raster imagery. .jpg produces lossy compression, which can result in a lessening of image quality. Vector graphic documents that are saved as .jpg files will be rasterized, resulting in a loss of their resolution independent scalability. There are two main reasons for using the .jpg format. The first use is to loose data. Now it sounds like this would be a bad thing, but in actuality, loosing data or shrinking a file can be very useful. For instance, you can create a high resolution illustration and retain an original file for many different uses. If one of those uses is to display that image on a website, you can compress the file by saving it in the .jpg format. The ,jpg format allows you to set the image quality, i.e., the image size, when you save your illustration, making it very small if needed. The second reason to use .jpg is that stores 24-bits per pixel, 16,777,216 colors for a realistic true color look. Most significantly, the .jpg format supports RGB, CMYK and grayscale color spaces.

JPEG low resolution color sample.

JPEG maximum resolution color sample.

EPS

EPS means “Encapsulated PostScript”. A format that was once quite popular, .eps encapsulates raster and vector image files in PostScript code. Postscript is a page description language in electronic and desktop publishing. It has a dynamic logic system based on rules where a programming language expressions denote functions developed by Adobe Systems. There are two main purposes for using an .eps format. One was to export an image that could be placed into another document, such as an illustration file that would be placed into a page layout program for final output. A placed .eps image retains its integrity; it can be scaled, but not resampled or content edited. .tif files can also be placed inside other documents and offer greater flexibility than .eps files. The other main purpose is to save a file in encapsulated PostScript to be output directly on a PostScript printer. .eps files can produce lossy compression in raster based applications like Photoshop.

PDF

PDF is the acronym for “Portable Document Format”.
 Based on PostScript, it was developed by Adobe Systems as a highly compressed, efficient way to format documents for platform independent readability. Another hallmark of a .pdf file is that it will look the same on screen as it does in print. The .pdf format has several quality variations available. For instance, Photoshop offers “Press Quality”, for high quality output on an imagesetter or platesetter, however, the file is CMYK down sampled to 300ppi for efficiency. Another offering is “Smallest File Size” where the file is down sampled to low resolution RGB for use on the web. .pdf is useful for saving page layout files with single or multiple pages with images,  text, fonts, graphics, and multimedia content. .pdf has grown greatly in popularity due to it’s extreme usefulness for viewing documents on all sorts of platforms like the iPad, iPhone, etc.

GIF

GIF is the abbreviation for “Graphics Interchange Format”. It is a raster image format that maxes out at 8-bits per pixel RGB, or 256 colors. At one time the .gif format was the most popular format for saving files for use on web documents. Due to its limitation on colors it is not generally used for full color illustrations or photography, although .gif is an excellent format for certain purposes such as illustrated icons and graphics with limited color schemes.

GIF 256 color sample.

GIF 256 color sample with maximum pixel dithering.

PNG
PNG stands for “Portable Network Graphics”. Preceded by .gif, the .png format has all but replaced .gif for use by illustrators when converting images for the web. .png is known for lossless compression in a raster image format. Another advantage over .gif is that the .png format comes in two sizes; PNG-8 or 8-bit per pixel, and PNG-24 or 24-bit per pixel; 256 colors and 16,777,216 colors. PNG-24 can handle full color illustrations and photographs while PNG-8 is better suited for limited color images. One limitation of .png is that it only supports the RGB color space. For this reason .jpg is the preferred format for full color images, but .png may be a better choice for illustrations that use a limited color pallet or solid colors.

PNG-8 color sample, 256 colors.

PNG-24 color sample, 16,777,216 colors.

SVG
The SVG file type is primarily associated with “Scalable Vector Graphics”. SVG is a language for describing two-dimensional shapes, text, and embedded raster graphics. Although not a natively supported by Adobe, SVG images can be created and exported from Adobe Creative Suite programs, such as Illustrator for use on the web. SVG is ideal for interactive, data-driven, personalized graphics. It is widely supported by modern web browsers.

Best Usage Summary

.tif – For CMYK illustrations for commercial printing

.tif – For RGB/CMYK illustrations for high quality desktop output

.tif – For CMYK illustrations placed in other application files

.jpg – For RGB/CMYK full color raster illustrations for desktop output or web

.eps – For illustrations for output on a PostScript device

.pdf – For illustrations for emailing, digital display or desktop output for proofing

.png – For illustrations with limited color for the web (PNG-8)

.png – For full color illustrations for the web (PNG-24)

.svg – For two-dimensional vector graphics for the web

One Final Word

For archiving I recommend using the native application file format and .tif. As long as applications are updated to be upwardly compatible, the native file format will be best. I use the .tif format for insurance.

Digital Color Spaces Defined

Digital visualization software offers a variety of color environments to suit the needs of illustrators and designers. Each environment has its own purpose and uses. Most of us are quite familiar with the color spaces we use. Still, a basic understanding of all of the color options can be useful.

RGB (The Monitor Model)

The RGB color model is composed of the primary colors of red, green, and blue. The model, also known as the transmitted model, is an additive color system. Color systems are classified by how the primary colors relate to white. An additive system means that when all three primary colors are added they create white.  When two of the primary colors are combined they produce a secondary color. In the RGB system the secondary colors are cyan, magenta and yellow. Color monitors and digital projectors use the RGB system to display color. According to the trichromatic, Young-Helmholtz theory of color vision, the RGB color system aligns very closely to the way the human eye processes color.

RGB color model. © 2012 Don Arday.

CMYK (The Print Model)

CMYK stands for cyan, magenta, yellow, and black. Not to be confused with the “B” in RGB, black is represented by “K”. Also referred to as a reflective model, the CMYK system is a subtractive color system. When CMY are combined they create black. White is produced by the total lack of color and a white paper surface. When two of the primary colors are combined they produce a secondary color. In the CMYK system the secondary colors are red, green and blue. This alignment creates compatibility between the CMYK and RGB systems when creating work for print output on a monitor system.

CMYK color model. © 2012 Don Arday.

HSV (The Color Theory Model)

HSV is the acronym for hue, saturation and value. This color model is also called HSB or hue, saturation and brightness; and HSL or hue, saturation, and lightness. Very familiar to illustrators, the HSV model is based on standard artists color theory. Although not specifically relating to display hardware or printing equipment, it is an available color model in many image software programs as a more intuitive system for use by artists.

The “hue” is a color. For instance, the color of sky on a clear day is a blue hue. Every color is a hue.

The “saturation”, also called chroma, is a variation in intensity of a color. It is the amount of grey proportionate to the hue. Saturation can be used to alter a hue.

The “value” is the amount of lightness or darkness in a color. Brightness can also be used to alter a hue.

HSV color model. © 2012 Don Arday.

Lab (The Color Opposition Model)

Lab is an acronym. The “L” stands for lightness; the “a” signifies the range of opposition between green and magenta; and the “b” signifies the range of opposition between blue and yellow. Starting at a neutral center greens are negative values while magentas are positive values. Similarly, blues are negative values and yellows are positive values. This complimentary color concept should seem familiar to illustrators and designers. A simpler way to understand Lab color is to think of warm colors as positives and cool colors as negatives. The type of Lab color used in computing is the CIELab model. The CIE stands for the International Commission on Illumination. The Lab system is based on the Munsell color system. Basically, the Lab and Munsell color spaces are extruded 3D versions of the familiar “color wheel”. Although an excellent model for helping artists to understand color, the Lab system is not as compatible as RGB or HSV for use on monitors or in software programming due to its complexity. The next time you are working with an image in Photoshop, switch over to the Lab color space just to see how surprisingly different the colors in your image appear.

Lab color model. © 2012 Don Arday.

Grayscale (The Luminosity Model)

The Grayscale model is simply a range of tones between black and none or white. The range in the grayscale model used by Adobe contains 255 shades of gray plus white, and it is an 8-bit mode.

Grayscale color model. © 2012 Don Arday.

Indexed Color (The Color Compression Model)

Like the 8-bit Grayscale model, an Indexed Color model is limited to a finite index of colors. There are several Indexed Color models in common use, including a 16 Color model, the PhotoImpact Optimization Palette, the Graphics Interchange Format palette, and others. The Standard Web Browser Palette with its limit of 216 colors is an Indexed Color model. The Index color space in Adobe Photoshop is limited to 256 colors. In the case of the Photoshop Indexed Color space, colors in a non-indexed color image are altered to match the 256 colors available. To achieve this each pixel color is shifted to the nearest color in the color palette and then dithered with similar colors to optically mix the non-available colors.

ANPA (American Newspaper Publishers Association) index of color
swatches.