Revibing Code from Papers: Reimplementing HCI Artifacts

Abstract

Software artifacts for most technical HCI research projects are unavailable. The lack of access to these imposes limits on academic knowledge production. It is difficult to: extend or reuse research artifacts; use strong baselines in evaluating follow-up work; and perform replication or reproducibility research. In this work, we demonstrate the potential of new agentic AI technologies to revibe interactive software: reimplement systems directly from research papers. To measure the success of the approach, we describe a revibeability metric. By revibing recent research papers from UIST, and interviewing their original authors, we demonstrate the plausibility (and limitations) of revibed system. The results are encouraging. In many cases producing code suitable for strong baseline use. We argue that this may represent a fundamental shift in how we produce, use, and evaluate research artifacts in the technical HCI community.