Segmentation of Dashboard Screen Images: Preparation of Inputs for Object-based Metrics of UI Quality

Using object-based metrics to analyse design aspects of user interfaces (UI) is a suitable approach for the quantitative evaluation of the visual quality of the user interfaces. Balance or Symmetry are examples of such metrics. On the other hand, we need to deal with the problem of the detection of objects within a user interface
screen which represent the inputs for the object-based metrics. Todays user interfaces (e. g., dashboards) are complex. They consist of several colour layers, and it is complicated to segment them by well-known page segmentation methods which are usually used for the segmentation of printed documents. We also need to consider the subjective perception of users and principles of objects grouping (as Gestalt laws). Users usually group simple objects (graphical elements and shapes) into coherent visually dominant objects. We analysed the experience of 251 users manually segmenting the dashboard screens to design a novel method for the segmentation of dashboard screen images. The method initially focuses on the reduction of image colours which represents image layers. Then, it detects the primitives which creates a screen layout. Finally, the method processes the screen layout using the combination of the top-down and bottom-up segmentation strategy and detects visually dominant regions.