How does real science actually get done? Help us record it.
We are building open datasets of scientific work — from single computational
tasks to entire multi-month research projects — so that AI agents can learn to
understand, assist, and eventually carry out research. Three complementary efforts
are recruiting participants and collaborators right now.
Across UMN: Chemistry · Biology · Medicine · Aerospace · Civil & Environmental · Materials Science · Electrical & Computer Eng. · Robotics · & moreThree time horizons: tasks → sessions → projects · Open data, co-authorship for contributing labs
Now recruitinggraduate students, lab members, and PIs across departments — refer an interested colleague or contribute your own lab's work.
One question, three scales of data
Each project captures scientific work at a different granularity. Together they span
the full arc of research — what a scientist does in an hour, and what unfolds over half a year.
They share a participant pool and recording infrastructure across departments.
If your lab works with simulators or computational tools, contribute well-specified tasks that become a public benchmark for AI agents on real scientific software.
What we collectA task description, its expected outcome, and the metrics that decide success — across chemistry, biology, aerospace, civil, and materials science.
Step 1
Scope
A short interview to scope the tasks your lab cares about.
Step 2
Submit
Specify each task as a YAML manifest: description, outcome, metrics.
Step 3
Benchmark
We run agents against your tasks and post results to a public leaderboard.
Co-authorshipAll participants are invited as co-authors
ProducesSciHarbor — a benchmark evaluating AI agents on real simulators (OpenFOAM, GROMACS, AlphaFold, Ansys, OpenRocket…). 300+ tasks in Phase 1, MIT-licensed, with an open leaderboard.Phase 2 extends to physical workflows once robotic arms are installed.
If your lab is running physical experiments, record short sessions so we can test how well vision-language models understand the goals behind each action — the User-Aware AI effort.
What we collectHead-mounted video with think-aloud narration; the participant's intent is annotated as the prediction target.
Step 1 · 30 min
Interview
Orientation, consent, instructions, and scheduling around your lab's calendar.
Step 2 · 2 hr
Recording
Record two ~1-hour sessions of the lab work you already had planned.
Step 3 · 30 min
Reflection
Interview about what you did, why you did it, and what you were aiming for.
Compensation$150 per participantup to $50 / hr · co-authorship for contributing labs
ProducesA dataset of hands-on lab workflows for benchmarking how well vision-language models connect the actions they observe to the scientific goals and intentions behind them.
Press play — fine-grained actions roll up into subtasks, which serve the session's goals, under the participant's cognitive state. Source video: GELFREE 8100 protein fractionation.
Related workSeparate efforts — not this project's page.Builds on SERUM, which extracts behavioral models from egocentric lab video (paper under review).
If you are running a long-term project in digital tools, let us record how it unfolds from inception to publication — tracking research trajectories in real time.
What we collectActivity from the tools where research actually happens — Overleaf, GitHub, Slack, Google Drive, Claude Code, and approved screen recordings — aligned into one project timeline.
Step 1 · 1 hr
Onboarding
Install SciFlowCollection and connect only the tools you approve.
Step 2 · 12 mo
Record & collect
Auto-records approved sessions and syncs your research files in the background.
Step 3 · 3 × 1 hr
Reflection
Interviews reviewing your recorded activity, reasoning, and decisions.
Compensation$150 per team / 4 monthsup to $600 total
ProducesA longitudinal dataset of how research unfolds — to study how early actions shape later ones, and to build AI research assistants that understand a project's whole history.
edited main.tex on OverleafDemo Participant · Jul 15, 11:24 AM
Everything shown is synthetic demo data. You choose exactly which folders, channels, repositories, and screens are shared — anything you don't approve is never collected. Full demo on the project page.
Related workSeparate efforts — not this project's page.Builds on ScholaWrite, a keystroke dataset of scientific writing.
Contribute your own scientific work — a couple of hours of lab recording, a
set of computational tasks, or passive recording of a project you're already running.
We handle setup, scheduling, and equipment.
For graduate students & lab members
Collaborate
Bring your lab into one or more of the projects. PIs help define what "correct"
looks like in their domain and open the door for their students to participate.
For PIs & research groups across departments
Co-authorship and contribution opportunities are available for participating PIs and lab members across all three projects.
Contributor PIs — collaborating faculty across UMN departments
Jihye Park
Chemistry · EC-MOFs
Chris Bartel
Materials Science · DMC Lab
Seung Hwan (Allen) Lee
Chemical Engineering · Enzymes
Seongjin Choi
Civil Engineering · Urban mobility AI
More domain experts across Chemistry, Chemical & Aerospace Engineering, Civil Engineering, Physics, and Mechanical Engineering are joining the initiative across all three projects.
