Learning Visual Importance for Graphic Designs and Data Visualizations

Zoya Bylinskii Nam Wook Kim Peter O'Donovan
Sami Alsheikh Spandan Madan Hanspeter Pfister
Fredo Durand Bryan Russell Aaron Hertzmann

[Paper][Supplement][Code][Video][Demo]


Video

This brief video showcases how our importance predictions can provide interactive feedback within a graphic design tool.


Demo

Upload a graphic design or a data visualization to obtain importance predictions using the single-image demo, or play with our interactive design tool with realtime feedback.

Single-Image
Our single-image demo allows you to upload either a graphic design or data visualization and get an importance prediction map for the image. This map estimates how likely people are to pay attention to a particular part of a design or visualization. You can choose the respective neural network model to pass the image to, and the model will predict in real-time.
Interactive
Our interactive demo allows you to move around, resize, and otherwise modify text and visual elements on a graphic design, and to receive updated, real-time predictions of importance. You can see how various design changes affect the predicted importance map, overlaid on top of the original design. Predictions of how users view a design can provide useful feedback.

Abstract

Knowing where people look and click on visual designs can provide clues about how the designs are perceived, and where the most important or relevant content lies. The most important content of a visual design can be used for effective summarization or to facilitate retrieval from a database. We present automated models that predict the relative importance of different elements in data visualizations and graphic designs. Our models are neural networks trained on human clicks and importance annotations on hundreds of designs. We collected a new dataset of crowdsourced importance, and analyzed the predictions of our models with respect to ground truth importance and human eye movements. We demonstrate how such predictions of importance can be used for automatic design retargeting and thumbnailing. User studies with hundreds of MTurk participants validate that, with limited post-processing, our importance-driven applications are on par with, or outperform, current state-of-the-art methods, including natural image saliency. We also provide a demonstration of how our importance predictions can be built into interactive design tools to offer immediate feedback during the design process.

Paper, code and citations

@inproceedings{predimportance, author = {Zoya Bylinskii and Nam Wook Kim and Peter O'Donovan and Sami Alsheikh and Spandan Madan and Hanspeter Pfister and Fredo Durand and Bryan Russell and Aaron Hertzmann}, title = {Learning Visual Importance for Graphic Designs and Data Visualizations}, booktitle = {Proceedings of the 30th Annual ACM Symposium on User Interface Software \& Technology}, year = {2017} }