Validating a User Interface (UI) has traditionally been a bottleneck. While we can easily automate functional tests to see if a button works, determining if that button is aesthetically correct or follows established design principles has always required a human expert. My current research aims to enhance the speed of human expertise by developing a Computer Vision-Assisted UI Validation Framework.
AIM
The primary goal is to design, develop, and evaluate a framework that uses Computer Vision (CV) to objectively assess User Interfaces based on established UI/UX theories. It specifically targets the current problem where UI validation is too resource-intensive and subjective.
OBJECTIVES
Translate Theory: Convert abstract principles like Visual Hierarchy, Gestalt Principles (Proximity/Similarity), and Color Theory into quantifiable metrics.
Build Prototype: Create a tool that can Parse elements (buttons, text), Evaluate them against rules, and Score the results.
Integration: Ensure the framework can be used within existing development workflows, such as CI/CD tools.
METHODS
Research through Design: A four-phase approach involving literature review, dataset construction, implementation, and evaluation.
Technical Stack: Utilizing Python with libraries like OpenCV and PyTorch, specifically employing deep learning models like YOLO for object detection.
Data Sourcing: Building a dataset from award-winning sites (e.g., Awwwards) to compare against “bad” design examples.
OUTCOMES
Objective Validation: An enhancement for subjective human observation to a scalable, automated visual inspection mechanism.
Correlation Findings: Determining the extent to which automated scores can successfully mirror human expert assessments.
Technical Insights: Identifying risks regarding whether high-level concepts like “minimalism” can be truly captured by algorithms.
OUTPUTS
The Workpiece: A software application or library that takes a URL or screenshot and generates a structured report.
Visual Overlays: Heatmaps or overlays on the UI that point out exactly where rules (like alignment or contrast) are violated.
Master’s Thesis: A formal document covering the translation gap between UX and Computer Vision.
