LawFlow

Research Overview

LawFlow captures full end‑to‑end legal workflows, revealing how law students navigate drafting tasks such as business‑entity formation—covering everything from client intake to filing. Unlike existing legal‑AI benchmarks that focus on isolated tasks, LawFlow records the entire decision process, including ambiguity, iteration, and client‑specific reasoning.
By comparing human and LLM‑generated workflows, LawFlow uncovers how human legal reasoning is modular, adaptive, and context‑sensitive—while LLMs tend to follow rigid, linear paths. These insights open new directions for building collaborative AI tools that support, rather than replace, human legal judgment.

Data Collection Tool

Our database captures real-world business entity formation scenarios, starting with a seed scenario provided by a local business. Through conversations with the business owner, a lawyer initiates the legal process, meticulously documenting and taking notes of each step—from initial discussions to formal filings. The process culminates in a finalized agreement document, serving as the definitive legal outcome.

Key Features:

📌 Subtask Annotation – Tag actions to modular task nodes from an expert-verified legal workflow. Supports non-linear paths and rationale logging.
📌 Live Note-Taking – Persistent field for informal notes (e.g., during client intake), with full keystroke logging for later analysis.
📌 Template Library – Access and reuse agreement templates, simulating real-world use of legal precedents and boilerplate search.
📌 Legal Search Interface – Perform open and template-specific searches, reflecting how legal professionals integrate research into decisions.
📌 Document Editor – Draft agreements in a built-in word processor; all edits are logged to capture revision behavior over time.

This data collection tool enables us to study not just what decisions are made while drafting by law students, but also how they adapt, iterate, and navigate ambiguity across the full drafting workflow.

Entity Formation Workflow

Our data collection tool is rooted in a visual, expert-informed task plan that maps out the full legal drafting workflow—from initial client intake to final agreement. 🎯 This task plan supports flexible navigation across subtasks in the data collection tool, while maintaining a clear structure rooted in real-world practice.

The workflow is organized across three levels of detail:

Inter-subtask level: Major phases like client intake, memo drafting, and agreement creation.
Intra-subtask level: Connected steps within each phase—e.g., gathering facts, conducting research, revising drafts.
Node level: Fine-grained individual actions, such as writing a clause or asking a follow-up question.

This multi-layered structure allows us to capture the dynamic, non-linear nature of legal work—supporting both detailed analysis and AI tools that align with how real practitioners think and operate.

For example, the entity formation workflow includes five key stages:

📌 Information Gathering: Identify client goals, legal gaps, and jurisdictional specifics.
📌 Deciding a Recommendation: Evaluate constraints, perform legal research, and determine a course of action.
📌 Drafting a Client Memo: Summarize recommendations, highlight open issues, and prepare for feedback.
📌 Drafting the Operating Agreement: Use precedent templates, edit iteratively, and align with client needs.
📌 Assessing Tax Treatment (Optional): Consider tax impacts and consult specialists when necessary.

Together, these levels form a flexible yet robust blueprint for capturing and analyzing legal workflows in real-world scenarios.

Generated Operating Agreements

Each scenario in our dataset is grounded in real-world business needs, with law students simulating the full legal workflow from client intake to final agreement drafting. Human workflows are flexible and iterative, often looping back to revise earlier steps. In contrast, LLMs follow a more rigid, linear path—highlighting key differences in how humans and AI approach complex legal reasoning.

Human Execution Graph

OpenAI O1 Execution Graph

Call for Participation

We plan to scale up our data collection and develop innovative legal reasoning benchmarks for evaluating long-range, multi-turn, deep legal reasoning tasks. We welcome collaborators, funders, and researchers interested in exploring AI for knowledge intensive tasks, such as legal processing. If you’d like to try our tool or contribute to our dataset, please get in touch! Contact us at dongyeop@umn.edu

LawFlow: Collecting and Simulating Lawyers' Thought Processes