Gabriel Johnson, Mark Gross, Jason Hong, and Ellen Do

Foundations and Trends in HCI

January 2009

Computational support for sketching is an exciting research area at the intersection of design research, human–computer interaction, and artificial intelligence. Despite the prevalence of software tools, most designers begin their work with physical sketches. Modern computational tools largely treat design as a linear process beginning with a specific problem and ending with a specific solution. Sketch-based design tools offer another approach that may fit design practice better. This review surveys literature related to such tools. First, we describe the practical basis of sketching — why people sketch, what significance it has in design and problem solving, and the cognitive activities it supports. Second, we survey computational support for sketching, including methods for performing sketch recognition and managing ambiguity, techniques for modeling recognizable elements, and human–computer interaction techniques for working with sketches. Last, we propose challenges and opportunities for future advances in this field.