Research
Technical reports and research publications on AI for creative workflows.
Image UI Grounding for Creative Workflows: Model and Benchmark
This report presents a model for image user interface (UI) grounding tailored to creative software, and introduces a new benchmark for evaluating AI understanding of creative workflows. Existing benchmarks do not adequately capture the complexity of creative software interfaces. We describe our model architecture, the construction of our benchmark, and report initial results.
Fine-tuning Vision-Language Models for Creative Software Understanding
We present a methodology for fine-tuning vision-language models specifically for creative software interfaces. Our approach adapts pre-trained VLMs to understand the unique visual and functional characteristics of design tools, video editors, and 3D modeling software. We demonstrate improved performance on creative workflow tasks compared to general-purpose models.
Tool Use in Adobe After Effects: A Framework for Creative Software Automation
We develop a framework for AI tool use in Adobe After Effects and demonstrate its applicability to other creative software like Figma and Blender. Our system enables AI assistants to perform complex creative tasks by understanding tool interfaces, parameter relationships, and workflow patterns. We evaluate performance across multiple creative domains and discuss generalization strategies.