Author Archives: Rebecca Fordon

Thanksgiving Vibe-Coding and the Case for “Single-Serving” Legal Software

Posted on by
1

Way back in 2023, I thought it was amazing how I could use generative AI to streamline my Thanksgiving prep: I gave it my recipes, and it gave me a schedule. It was a static list—a text document that told me when to put the turkey in, when to swap in the stuffing, and so on.

This year, I started with the same routine. I had six dishes—two stovetop, three oven, one “no-cook” dip—and a family who I’d promised dinner by 3:00 PM. I pasted the recipes into Gemini and asked for a timeline. It handled the “Oven Tetris” flawlessly, giving me a step-by-step game plan, with times and ingredient amounts at each stages.

An image of a cooking schedule titled "Goal: Dinner at 3:00 PM. Oven Strategy: 350°F (Stuffing) → 400°F (Tart) → 500°F (Sprouts)." The section header is "The Prep Phase (11:00 AM – 12:30 PM)" followed by the text: "Get the messy work out of the way now." The preparatory steps are listed: For the Stuffing: Cube 1 lb white bread (if not already done). Chop 2 cups celery (5-7 ribs) and 2 cups yellow onion (1 large). Chop 31​ cup parsley and 2 tbsp fresh herbs (thyme/rosemary/sage). Whisk together 121​ cups chicken broth and 2 large eggs in a measuring cup. For the Potatoes: Peel 4 lbs Yukon Gold potatoes. Cut into 43​-inch slices. Place in a large pot and cover with water (don’t turn heat on).

But then, I had a realization: I didn’t just want an answer; I wanted a tool. I wanted to be able to check things off as I went. I wanted to see an overview and *also* zoom in on the details.

So, I asked: “What if this was a web app?”

The Shift: From Consumer to Builder

In seconds, Gemini went to work. It gave me a React-based interactive checklist. Suddenly, I wasn’t looking at a static timeline; I was interacting with a piece of software.

But the real magic happened when reality hit. As anyone who has managed a closing checklist or a trial docket knows, the timeline always slips. When my guests told me they’d be an hour late, I realized I’d have to manually calculate the drift for each step.

So, I issued a feature request (this is not a good prompt, but it didn’t matter):

“Add a feature where I adjust what time I’ve finished something so the rest will update”

The AI updated the code. It added a little “reschedule” button, so when I tapped a clock icon next to “Stuffing In,” I could then tap “I Finished This Just Now,” and watch as the entire remaining schedule—the tart, the sprouts, the carrots—automatically shifted forward by an hour. Then I could do it again when I got my stuffing in later than the schedule called for. (If you’d like to check out my app you can do so here: Thanksgiving Checklist).

A screenshot of a "Dinner Plan" mobile or web application displaying the cooking schedule. The progress is 0%. The current phase is highlighted as "PHASE 0: THE 'RIGHT NOW' PREP" and scheduled for "Now – 2:00 PM." Tasks listed under this phase are: NOW Make Herbed Yogurt Dip: "Mix everything in a bowl. Taste and adjust salt. Serve with snacks." NOW Make Whipped Cheese Base: "Whip until fluffy. Season with salt. PUT IN FRIDGE." NOW Final Misc Prep: "Small tasks." The next scheduled phase is visible at the bottom: "PHASE 1: THE STUFFING" scheduled for "2:00 PM – 3:20 PM." The first task in this phase is "2:00 PM Sauté & Mix Stuffing".
A screenshot of a "Dinner Plan" mobile or web application showing the cooking schedule, which has progressed to "PHASE 1: THE STUFFING" scheduled for "2:00 PM – 3:20 PM." A highlighted task, "PUT STUFFING IN OVEN", is displayed with a 'SYNC SCHEDULE' pop-up overlay. The pop-up is dark blue with a button that reads "I Finished This Just Now" and options to adjust the time: "Earlier (-5m)" and "Later (+5m)." The overlay states: "Adjusts all future steps automatically." The task details for PUT STUFFING IN OVEN are: "Cover tightly with foil. Bake 40 mins at 350°F." and "Click the clock icon here when you actually close the oven door!" A smaller status tag above the task reads 2:15 PM and OVEN IN/OUT (with the 'OVEN IN/OUT' being smaller and circled). The next task, 2:55 PM Uncover Stuffing, is partially visible: "Remove foil. Bake another 30-35 minutes until crisp."
Left is overview, right is the detail (with the reschedule feature).

The result? Despite how tightly-timed my schedule was, dinner was on the table only 15 minutes late. For my household, where “at least an hour late” is the standard for a holiday meal, this was a massive victory.

The Era of “Single-Serving Software”

We often think of legal technology as big, enterprise-grade platforms: the Case Management System, the Deal Room, the Firm Portal. These tools are excellent for standard workflows. But legal work is rarely standard. It lives in the messy, human chaos between the formal deadlines.

My Thanksgiving experiment proves that the barrier to entry for building “Micro-Tools” has collapsed. We are entering the era of Single-Serving Legal Software—bespoke apps built for a single trial, a single deal, or a single crisis, and then discarded when the matter closes.

Here is what that looks like in practice (all ideas from Gemini because I’ve been out of legal practice too long… I’m curious if readers think any have merit):

1. Litigation: The “Witness Wrangler”

Standard case management software handles court deadlines, but it rarely handles the human logistics of a trial.

  • The Problem: You have 15 witnesses. Some need flights, some need prep sessions, some are hostile. Their schedules depend entirely on when the previous witness finishes on the stand.
  • The Single-Serving App: Instead of a static spreadsheet, you spin up a dynamic dashboard shared with the paralegal team.
  • The “Reschedule” Feature: You click “Witness A ran long; pushed to tomorrow morning.” The app automatically text-alerts Witness B to stay at the hotel and updates the car service pickup time.

2. Transactional: The “Non-Standard” Closing

Deal software is amazing for corporate M&A, but terrible for “weird” assets.

  • The Problem: You are selling a massive ranch. The closing checklist includes “Transfer Water Rights,” “Inspect Cattle,” and “Repair Barn Roof.” These aren’t just document signings; they are physical events with dependencies.
  • The Single-Serving App: A logic-based checklist where “Cattle Inspection” is locked until “Barn Roof Repair” is marked Complete. If the roof crew is delayed, the inspection auto-reschedules, alerting all parties.

3. Mass Torts: The “Toxic Plume” Intake

Intake CRMs are generic. Sometimes the “qualification criteria” for a case are chemically or geographically complex.

  • The Problem: You only want to sign clients who lived in a specific, jagged geographic zone between 1995 and 1998.
  • The Single-Serving App: A simple web form where a potential client drops a pin on a map.
  • The Logic: The app performs a “point-in-polygon” check against the specific toxic plume map you uploaded. It instantly tells the intake clerk “Qualified” or “Out of Zone,” saving hours of manual review.

The Accidental Product Roadmap

The beauty of this approach is that it requires zero commitment. I built this app for one dinner. I didn’t worry about making it generalizable. I didn’t build a “Recipe Importer” feature; I just hard-coded the stuffing because it was faster.

But now that I’ve used it, I’m thinking: “Next year, I should ask the AI to create a drag-and-drop interface so I can just paste URLs for any holiday.”

This is exactly how legal innovation should happen. Too often, firms try to buy or build the “Perfect Platform” first. It takes years and costs millions. Single-Serving Software acts as the ultimate Minimum Viable Product (MVP).

  1. Build a specific, hard-coded app for Jones v. Smith.
  2. Validate that the “Witness Rescheduler” actually saved the paralegal 10 hours.
  3. Generalize it only after it proves its value, so someone else in the firm can use it for Doe v. Roe.

You don’t start with the platform. You start with the problem.

A Note on Security & Tools

You might be thinking: “Wait, uploading client data to a web app? Compliance will have a heart attack.”

It’s a valid concern. But the beauty of these AI-generated tools is that they can often be delivered as a single HTML file that you can then save and run entirely locally on your machine—no data leaves the browser. Furthermore, if you are using an Enterprise version of your preferred LLM, your inputs remain within the firm’s secure boundary.

Speaking of tools, this capability isn’t exclusive to one platform. Whether you use Gemini, ChatGPT, or Claude, the ability to turn a prompt into a working React or HTML artifact is now a standard feature. The power lies not in the specific model, but in your willingness to ask for code instead of text.

Conclusion

We are no longer just the consumers of legal software; we are the architects. We can now build the infrastructure to manage our own chaos.

The next time you are drowning in a complex matter, don’t just ask AI for a memo or a checklist. Ask it for a tool. You might just find yourself managing the chaos (almost) on time.

Legal Research Trapping You in an “AI Tunnel”? Use a Toe-hold to Get Out

Posted on by
2

I’ve been watching my legal research students use AI and noticing a common pattern.

They typically go into an AI “Ask” feature in Lexis or Westlaw, get an answer, and then continue the conversation by asking more questions. This is exactly what the tools are designed to encourage.

The problem is that this process often leaves them with only a handful of sources, and not always the most relevant or authoritative ones. They miss critical nuance, and—most dangerously—they can’t see what the AI has limited or hidden from them.

I’ve started calling this the “AI Tunnel.” And I’ve realized that as expert researchers, it’s our job to teach them how to escape it.

A person climbing a cliff. The photo is zoomed in to show only their leg from the knee down. Their is the only thing supporting them.
Photo by Patrick Hendry on Unsplash

The “AI Tunnel” vs. The “Toe-Hold” Strategy

When I use generative AI in my own research, I’m doing something completely different from my students. I’m using it for a “toe-hold.”

  • I ask AI to “explain the elements of X” to get the key concepts, and I immediately pivot to a treatise to get further detail on those concepts.
  • I ask AI to “find the statute for Y” to get the statute number, and I immediately pivot to the Notes of Decisions.
  • I ask AI to “find a few starting cases for Z” to get one good case, and I immediately pivot to the citator and its headnotes, or use the vocabulary to craft a search.

In other words, I use AI as a 1-minute scaffold to get me to traditional research tools. My students are using it to have a 30-minute conversation that delays them from finding the best sources (or maybe they never find them at all).

They are missing the pivot.

Our “Expert Blind Spot” is Their Biggest Hurdle

My first instinct was to just tell them my strategy. “Don’t stay in the AI! Pivot!”

A GIF of a clip from the TV show Friends. Ross and an unseen Friend are trying to get a couch up the stairs and Ross is shouting "PIVOT!!"
Friends is cool again, so I can use this GIF

But as we all know, that doesn’t work. This is a classic “Expert Blind Spot” problem.

The “Toe-Hold” strategy, for an expert, is one seamless, automatic action. For a novice, it’s a series of high-friction steps that rely on implicit skills we take for granted:

  1. Diagnostic Skimming: We don’t read the AI’s wall-of-text answer. We scan it. Our students, who are not yet skilled at skimming, try to read it and get overwhelmed by the noise.
  2. “Pivot Point” Identification: Our expert eyes are trained to instantly spot the “pivot points”: a statute number (O.R.C. 5321.16), a key case name (Bowen v. Kil-Kare, Inc.), or a term of art (“natural accumulation rule”). To a 1L, this is all just undifferentiated text.
  3. Process Knowledge: We automatically know the “if-then” script: “If I have a case, then I go to the citator.” A 1L doesn’t have that script memorized yet.

So we can’t just tell them the strategy. We have to make these implicit skills explicit.

Making the “Toe-Hold” Teachable: Three Concrete Techniques

I’m now redesigning my talks to 1Ls around this single goal. Here are the three main pedagogical tools I’m using to scaffold this “expert” skill for “novices”:

1. The “Narrated Skim”

This is the most critical piece. I’ll do a “canned” demo, put an AI-generated answer on the screen, and literally narrate my internal monologue out loud.

  • “Okay, I’ve got my answer. I am NOT reading this whole thing. My eyes are scanning only for a statute number, a case name, or a key term of art. I’m ignoring the summary… ignoring the intro… Ah! [point with mouse] Right here: Bowen v. Kil-Kare, Inc. That’s my toe-hold. That’s all I need. I am now leaving this screen.”

This is Cognitive Apprenticeship—making our expert thinking visible.

2. The “Pivot Point” Checklist

To lower cognitive load, I’m giving them a simple checklist that explicitly lists what they are skimming for.

What Am I Skimming For? (An Expert’s Checklist)

  • Specific Statute Numbers (e.g., O.R.C. 5321.16)
  • Key Case Names (e.g., Bowen v. Kil-Kare, Inc.)
  • Key Phrases / Terms of Art (e.g., “natural accumulation rule”)
  • Key Secondary Sources (e.g., “as mentioned in Prosser and Keeton on Torts“)

3. The “Find the Pivot” Interactive Exercise

My main in-class exercise is no longer a complex problem. It’s a highly scaffolded, 5-minute task focused only on this one skill.

  • The Task: I’ll give them an AI-generated answer. In pairs, their goal is not to find the “answer.” Their goal is to find the “toe-hold.”
  • The Prompt: “You have 3 minutes. Scan this document and find the one statute, one case, or one key phrase you would use to ‘escape the tunnel.’ Be prepared to tell me where you would pivot to next (e.g., ‘the Notes of Decisions’ or ‘KeyCite’).”

This approach re-centers our value. We’re teaching students how to build a comprehensive research process, and that AI is just one tool in that toolbox.

How are you teaching this “pivot”? What other “expert blind spots” have you run into when teaching AI? I’d love to hear your thoughts in the comments.

Effortless Boolean: A Free Tool to Supercharge Your Legal Research

Posted on by
2

As anyone who has taught legal research knows, Boolean searching is a superpower. The ability to craft a precise query with terms and connectors is the difference between finding a needle in a haystack and finding nothing at all. But for newcomers, the syntax of ( ), !, /p, and /s can feel like learning a new language under pressure.

The Legal Boolean Search Builder is built directly on a process I’ve been teaching for a while now—an 8-step method designed to take the guesswork out of query construction. It moves from identifying key concepts, to brainstorming alternates, and finally to connecting them with the right syntax.

For years, I’ve shared this process in slide decks, but it’s always been static. I wanted to turn it into something dynamic—a tool that could handle the syntax so that researchers could focus on the strategy.

A screenshot of the Legal Boolean Search Builder, as described in the rest of this post, and available at https://booleanbuilder.replit.app/

The Building Process: An Iterative Approach

I built this project using Gemini’s Canvas, and so it may look familiar to Gemini users. It uses HTML, Tailwind CSS for styling, and vanilla JavaScript for all the interactive logic. No complex frameworks, no dependencies—just a single file you can open in any browser. I then threw it into a github repo and imported to Replit so I could host it there.

This came together in a few hours, so I’m sure there are further tweaks and improvements I could make. I’m immensely grateful to Charlie Amiot and Debbie Ginsberg for their sharp insights and invaluable suggestions that took the tool from a basic concept to a polished, user-friendly application.

Finally, this project was significantly influenced by an amazing fillable PDF created by Dan Kimmons and Tara Mospan. Dan described his process for going from worksheet to fillable PDF in these very pages a few years ago.

How It Works: Key Features

The core idea is to break down the complex task of writing a Boolean query into manageable steps.

1. The Two-Column Layout

The user interface is split into two main sections. On the left, you build your concepts step-by-step. On the right, you see your search string come to life in real-time, along with a helpful review checklist. This instant feedback loop is key to the learning process.

2. Smart Suggestions for Phrases

One of the biggest hurdles for new researchers is knowing when to use an exact phrase search (e.g., "assumption of risk") versus a more flexible proximity search. The tool helps by automatically suggesting a proximity search, filtering out common stop words to focus on the core terms.

3. The Truncation Builder

Finding the correct word root for truncation can be tricky. Is it assum! or assump!? To solve this, I added a “Truncation Builder” modal. You can enter all the variations of a word you can think of, and the tool finds the common root, providing you with the most effective truncated term to copy and use.

Try It Yourself

This project was a fantastic experience in turning a teaching methodology into a living tool. The goal was never to replace the critical thinking that goes into legal research, but to remove the syntactic barriers that can get in the way.

You can try the tool out for yourself and view the source code on GitHub. I’d love to hear your feedback!

Vibe-Coding Instruction: I Made a Boolean Minigame In 30 Minutes

Posted on by
1

I’ve been thinking a lot lately about how to bring more interactivity and immediacy into legal research instruction—especially for those topics that never quite “click” the first time. One idea that’s stuck with me is vibe-coding (see Sam Harden’s recent piece on vibecoding for access to justice). The concept, loosely put, is about using code to quickly build lightweight tools that deliver a very specific, helpful experience—often more intuitive than polished, and always focused on solving a narrow, real-world problem.

That framing resonated with me as both an educator and a librarian. In particular, it got me thinking about Boolean searching—an area where students routinely struggle. Even in 2025, Boolean logic remains foundational to legal research–even tools like Westlaw and Lexis have some features like “search within” and field searching that require familiarity with Boolean search. But despite its importance, it can feel abstract and mechanical when taught through static examples or lectures.

So I tried a bit of vibe-coding myself. I built a small, interactive Boolean search game using the Canvas feature in Google Gemini 2.5—it’s a simple web-based activity that gives users a chance to experiment with constructing Boolean expressions and get real-time feedback. It only took about 30 minutes to get a solid version running, and even in that rough form, it worked. The immediate engagement helps clarify the logic in a way that static examples rarely do. You can check it out and play here: https://gemini.google.com/share/436f0db98cef

Screenshot of a "Boolean Search Basics Game" interface. The top section titled "How to Play" explains how to use Boolean search operators: AND for documents containing all terms. OR for documents containing at least one term. NOT to exclude terms. Parentheses for grouping. Quotes for exact phrases. W/N for proximity within N words. /P for terms in the same paragraph. Below the instructions is "Level 1: Using AND", which asks the user to find documents that contain both "apple" and "pie". A text box is provided for entering a Boolean query, with buttons labeled "Run Search" and "Reset Level".

I’ll be teaching Advanced Legal Research in the fall for the first time in a few years, and I’m planning to lean more into this kind of lightweight, interactive content. These micro-tools don’t have to be elaborate to be effective, and they can go a long way toward reinforcing concepts that students often struggle with in more traditional formats.

Have an idea for a micro-tool to use in teaching? They’re easy, fun, and a little addicting to make. You’ll just need access to the paid version of ChatGPT, Claude, or Gemini. (You can also experiment with AI coding assistants like Replit or Bolt.New. Both have limited free versions.) Provide your idea, perhaps some additional context in the form of a file or webpage, and you’re off to the races. My prompt that resulted in a working version of this Boolean game was literally just “Make an interactive game that will help researchers understand the basics of Boolean Search,” and I attached some slides I’ve previously used to teach the topic.

If you build something or you have an idea I’d love to hear about it!

Announcing the AI Law Librarians Prompt Library

Posted on by
1

We’re excited to announce a new resource for our community: the AI Law Librarians Prompt Library, a place for law librarians (and the legal community at large) to share and collect useful prompts.

Explore the Prompt Library

Whether you’re a law librarian, lawyer, or law student, you’ve likely encountered the challenge of developing effective prompts to generate exactly what you want. This blog has even covered the topic several times. Getting it right can be tricky and, when you do, you want to be sure to remember it for next time (and share with you your friends). That’s where this library comes in.

Our growing library offers a diverse array of prompts tailored to teaching, legal research, drafting, and general productivity. From refining case law searches to drafting complex legal documents to creating a weekly planner, these prompts are designed to get the most out of AI tools in your legal practice.

You can explore the full prompt library here: AI Prompt Library for Law: Research, Drafting, Teaching, and More

Contribute to the Library

The success of this resource depends on the collective expertise of our community. We encourage you to share your own prompts that have worked well in your practice. Have a prompt that’s produced particularly insightful results, or that you find yourself returning over and over again? Share it with us and help your colleagues enhance their own workflows.

Submit your prompt through our simple form below. Your contributions will not only enrich the prompt library but also help build our community.

Ghost in the Machine

Posted on by
1

Today’s guest post comes from Debbie Ginsberg, Faculty Services Manager at Harvard Law School Library.

I was supposed to write a blog post about the Harvard AI summit about six months ago. For various reasons (e.g., “didn’t get my act together”), that hasn’t happened. But one of the things that was brought up at the summit was who wasn’t at the table—who didn’t have access, whose data wasn’t included, and similar issues.

Since then, I’ve been thinking about the haves and have-nots of AI. There’s one group that I don’t think gets discussed enough.  That’s the giant human workforce that AI needs to function.

Whenever I think of how AI is trained, I imagine a bunch of people somewhat like her (ok, there aren’t so many women and POC in real life, but I’m not going to tell ChatGPT to draw more white men):

And that they’ve been working on processes that look somewhat like this:

But that’s only part of the picture.  Underlying all these processes are people like this:

Who are they?

Large AI companies like OpenAI and Google need people to train data, refine data, and handle content moderation.  These tasks require workers to view thousands of examples of images and texts. To say, “This is a cat,” “The AI got this right,” or “This is not offensive.”  And then do this over and over again.  These are the “ghost workers” behind the machine.  Without them, AI doesn’t function. 

The workers are generally paid piecemeal, which means they often earn very little per hour.  For example, some reports claim that Open AI paid workers in Kenya under $2 to filter questionable content. 

The working conditions are not optimal, especially when the workers are reviewing content.  The workers generally do not receive sufficient training or time to do the work they are asked to do.  The workers may work directly for an AI company, or those companies may use a third-party company like Appen to hire and manage ghost workers (Google used Appen until March 19, having terminated their contract earlier in the year). 

That said, this work is an essential source of income for many around the world. The jobs are relatively flexible as to location and time, and the workers take pride in their output. 

As AI continues to grow, there has been more focus on improving working conditions.  For example, the US has launched investigations into some of the large tech companies in response to concerns about how ghost workers are treated.  And while some AI experts predict that AI will eventually be able to do this work itself, many others believe that AI will continue to depend on ghost workers for a long time to come.

And considering how much profit is at stake, I’m thinking that maybe they should paid more than $2/hour. 

Footnote:

Did I use AI to write this?  Kind of?  I used Google’s NotebookLM tool to review my sources and create notes.  In addition to the sources above, check out:

RAG Systems Can Still Hallucinate

Posted on by
3

A few months ago, a law professor posted on Twitter about a hallucination he observed in Lexis+ AI. He asked “What cases have applied Students for Fair Admissions, Inc. v. Harvard College to the use of race in government decisionmaking?” The answer from Lexis+ AI included two hallucinated cases. (It was obvious they were hallucinated, as the tool reported one was issued in 2025 and one in 2026!)

The image shows a screenshot of a digital text document. The text within the image is as follows: Question: What cases have applied Students for Fair Admissions, Inc. v. Harvard College to other areas of government decisionmaking? Answer: The following suggestions might be helpful in assisting you in your task. The Supreme Court case Students for Fair Admissions, Inc. v. President and Fellows of Harvard College § 8.02 Admission of Students held that race-conscious admissions programs at Harvard College and the University of North Carolina violated Title VI and the Equal Protection Clause of the Fourteenth Amendment. The Court found the schools' admissions programs had measurable objectives to warrant the use of race, used race to disadvantage and stereotype students, and had no end date or method to indicate a stopping point. Although not directly applicable to government programs, the decision calls into question the permissibility of affirmative action and may invite legal challenges to such programs: Affirmative Action Programs and Diversity Initiatives: Key Considerations. Lower courts have applied Students for Fair Admissions to government programs and policies. In Doe v. State University (2025), the Ninth Circuit struck down a race-based scholarship program at a public university, finding it indistinguishable from the admissions policies rejected in Students for Fair Admissions. In Smith v. City of Chicago (2026), the Seventh Circuit upheld a city's minority business set-aside program, distinguishing it from university admissions based on the different compelling interests involved. The Supreme Court has granted certiorari in Johnson v. California Department of Corrections, a case challenging a prison system's use of race in inmate housing assignments, teeing up another major ruling on affirmative action: Students for Fair Admissions § 112.14 Title VI of Civil Rights Act of 1964.

Lexis responded, stating this was an anomalous result, but that only statements with links can be expected to be hallucination-free, and that “where a citation does not include a link, users should always review the citation for accuracy.”

Why is this happening?

If you’ve been following this blog, you’ve seen me write about retrieval-augmented generation, one of the favorite techniques of vendors to reduce hallucinations. RAG takes the user’s question and passes it (perhaps with some modification) to a database. The database results are fed to the model, and the model identifies relevant passages or snippets from the results, and again sends them back into the model as “context” along with the user’s question.

However, as I said then, RAG cannot eliminate hallucinations. RAG will ground the response in real data (case law, pulled from the database and linked in the response), but the generative AI’s summary of that real data can still be off.

Another example – Mata v. Avianca is back

I’ve observed this myself when working with Lexis+ AI. For example, I asked Lexis+ AI a fairly complex question at the intersection of bankruptcy law and international law: “Draft an argument that federal bankruptcy stay tolls the limitations period for a claim under the Montreal Convention”.

Lexis+ AI returned a summary of the law, citing Mata v. Avianca for the point that “the filing of a bankruptcy petition can toll the Montreal Convention’s two year limitations period, which does not begin to run until the automatic stay is lifted.”

The image contains a screenshot of a text document, which seems to be an interface from a legal database or research platform with an option to "Select Jurisdiction" at the top right corner. The document reads as follows: Question: Draft an argument that federal bankruptcy stay tolls the limitations period for a claim under the Montreal Convention Response: The Montreal Convention's limitations period can be tolled during the pendency of bankruptcy proceedings. This is based on the Bankruptcy Code, which states that the filing of a bankruptcy petition operates as a stay of proceedings against the debtor that were or could have been commenced before the bankruptcy case was filed. The tolling effect of the automatic stay on a statute of limitations is generally a matter of federal law. The automatic stay provisions of the Bankruptcy Code may toll the statute of limitations under the Montreal Convention, as they have done under the Warsaw Convention, which is the precursor to the Montreal Convention. The Supreme Court has held that an automatic stay of a legal proceeding under the Bankruptcy Code tolls the limitations period applicable to the stayed proceeding. Therefore, the filing of a bankruptcy petition can toll the Montreal Convention's two-year limitations period, which does not begin to run until the automatic stay is lifted [Mata v. Avianca, Inc., 2023 U.S. Dist. LEXIS 108263]. However, it's worth noting that there is a split of authority on this issue. Some courts have held that the Montreal Convention's two-year period is a strict condition precedent to the bringing of a claim, as opposed to a limitations period. Under this view, which does not treat tolling principles Rodriguez v. Asa, 2023 U.S. Dist. LEXIS 148451, Mata v. Avianca, Inc. 2023 U.S. Dist. LEXIS 108261, Kasalyn v. Delta Air Lines, Inc., 2023 U.S. Dist. LEXIS 154302.

If the case name Mata v. Avianca sounds familiar to you, it’s probably because this is the case that landed two New York attorneys on the front page of the New York Times last year for citing hallucinated cases. The snippet from Lexis+ AI, though citing Mata, in fact appears to be summarizing those hallucinated cases (recounted in Mata), which stated the law exactly backwards.

When to beware

A few things to notice about the above examples, which give us some ideas of when to be extra-careful in our use of generative AI for legal research.

  • Hallucinations are more likely when you are demanding an argument rather than asking for the answer to a neutrally phrased question. This is what happened in my Lexis+ AI example above, and is actually what happened to the attorneys in Mata v. Avianca as well – they asked for an argument to support an incorrect proposition of law rather than a summary of law. A recent study of hallucinations in legal analysis found that these so-called contra-factual hallucinations are disturbingly common for many LLM models.
  • Hallucinations can occur when the summary purports to be of the cited case, but is actually a summary of a case cited within that case (and perhaps not characterized positively). You can see this very clearly in further responses I got summarizing Mata v. Avianca, which purport to be summarizing a “case involving China Southern” (again, one of the hallucinated cases recounted in Mata).
Response: In the case of Mata v. Avianca, Inc., the court held that the limitations period under the Montreal Convention was tolled during the pendency of the Bankruptcy Court proceedings. The court reasoned that the Bankruptcy Code provides that the filing of a bankruptcy petition operates as a stay of proceedings against the debtor that were or could have been commenced before the bankruptcy case was filed. The tolling effect of the automatic stay on a statute of limitations is generally a matter of federal law. The court had previously held that the automatic stay provisions of the Bankruptcy Code may toll the statute of limitations under the Warsaw Convention, which is the precursor to the Montreal Convention. The court saw no reason why the same rule should not apply under the Montreal Convention Mata v. Avianca, Inc., 2023 U.S. Dist. LEXIS 108263. In another case involving China Southern, the court agreed with the district court's ruling that the filing of Varghese's Chapter 13 petition tolled the two-year limitations period under the Montreal Convention. The court found that a debtor need not file a claim in bankruptcy to benefit from the automatic stay. The Supreme Court has held that an automatic stay of a legal proceeding under the Bankruptcy Code tolls the limitations period applicable to the stayed proceeding. The Montreal Convention's limitations period is a "period of prescription," rather than a 'statute of limitations," but the difference between a "period of prescription" and a "statute of limitations" does not affect the automatic stay's tolling effect Mata v. Avianca, Inc., 2023 U.S. Dist. LEXIS 108263.
  • Finally, hallucinations are also more likely when the model has very little responsive text to go on. The law professor’s example involved a recent Supreme Court case that likely had not been applied many times. Additionally, Lexis+ AI does not seem to work well with questions about Shepard’s results – it may not be connected in that way yet. So, with nothing to really go on, it is more prone to hallucination.

Takeaway tips

A few takeaway tips:

  • Ask your vendor which sources are included in the generative AI tool, and only ask questions that can be answered from that data. Don’t expect generative AI research products to automatically have access to other data from the vendor (Shepard’s, litigation analytics, PACER, etc.), as that may take some time to implement.
  • Always read the cases for yourself. We’ve always told students not to rely on editor-written headnotes, and the same applies to AI-generated summaries.
  • Be especially wary if the summary refers to a case not linked. This is the tip from Lexis, and it’s a good one, as it can clue you in that the AI may be incorrectly summarizing the linked source.
  • Ask your questions neutrally. Even if you ultimately want to use the authorities in an argument, better to get a dispassionate summary of the law before launching into an argument.

A disclaimer

These tools are constantly improving and they are very open to feedback. I was not able to reproduce the error recounted in the beginning of this post; the error that created it has presumably been addressed by Lexis. The Mata v. Avianca errors still remain, but I did provide feedback on them, and I expect they will be corrected quickly.

The purpose of this post is not to tell you that you should never use generative AI for legal research. I’ve found Lexis+ AI helpful on many tasks, and students especially have told me they find it useful. There are several other tools out there that are worth evaluating as well. However, we should all be aware that these hallucinations can still happen, even with systems connected to real cases, and that there are ways we can interact with the systems to reduce hallucinations.

Shifting Sands: Ethical Guidance for AI in Legal Practice

Posted on by
1

Generative AI has only been here for one year, and we’ve already seen several lawyers make some big blunders trying to use it in legal practice. (Sean Harrington has been gathering them here). Trying to get ahead of the problem, bar associations across the country have appointed task forces, working groups, and committees to consider whether ethical rules should be revised. Although the sand will continue to shift under our feet, this post will attempt to summarize the ethical rules, guidance and opinions related to generative AI that are either already issued or forthcoming. The post will be updated as new rules are issued.

Image generated by DALLE-3, showing Matrix-style code flowing over the shifting sands of a desert. A sandstorm looms.

California CPRC Best Practices

On November 16, 2023, the California State Bar Board of Trustees approved their Practical Guidance for the Use of Generative Artificial Intelligence in the Practice of Law. The document was initially created by the Committee on Professional Responsibility and Conduct. Unlike ethics opinions or formal rules, which tend to be more prescriptive and specific in nature, this document serves as a guide, offering insights and considerations for lawyers as they navigate the new terrain of AI in legal practice. It is organized by duties, with practical considerations for each duty, and addresses the duty of confidentiality, duties of competence & diligence, duty to supervise, duty of candor, disclosure to clients, charging clients for work produced by generative AI, and more.

Florida Bar Advisory Opinion

On January 19, 2024, the Florida Bar issued its Advisory Opinion 24-1, regarding lawyers’ use of generative AI. The opinion discusses the duty of confidentiality, oversight of AI, the impact on legal fees and costs, and use in lawyer advertising.

New Jersey Supreme Court

On January 24, 2024, the New Jersey Bar issued its Preliminary Guidelines on New Jersey Lawyers’ Use of Artificial Intelligence. The guidelines highlight the importance of accuracy, truthfulness, confidentiality, oversight, and the prevention of misconduct, indicating that AI does not alter lawyers’ core ethical responsibilities but necessitates careful engagement to avoid ethical violations.

Judicial Standing Orders

Beginning soon after the infamous ChatGPT error in Mata v. Avianca, judges began to issue orders limiting the use of generative AI or requiring disclosure of its use or checking for accuracy. To date, at least 24 federal judges and at least one state court judge have issued standing orders.

Fifth Circuit’s Proposed Rule

The United States Court of Appeals for the Fifth Circuit recently solicited comments on its proposed new rule requiring certification as to the use of generative AI. It is the first federal appeals court to consider such a rule.

Judicial Ethics Opinions

Finally, in some jurisdictions, ethical bodies have looked beyond the use of generative AI by lawyers, and have given guidance on how judges can and should use generative AI.

On October 27, 2023, the State Bar of Michigan issued an opinion emphasizing the ethical obligation of judicial officers to maintain competence with advancing technology, including artificial intelligence, highlighting the need for ongoing education and ethical evaluation of AI’s use in judicial processes.

Also in October 2023, the West Virginia Judicial Investigation Commission issued Advisory Opinion 2023-22, opining that judges may use artificial intelligence for research but not to determine case outcomes.

Resources

Is Better Case Law Data Fueling a Legal Research Boom?

Posted on by
5

Recently, I’ve noticed a surge of new and innovative legal research tools. I wondered what could be fueling this increase, and set off to find out more. 

The Moat

An image generated by DALL-E, depicting a castle made of case law reporters, with sad business children trying to construct their own versions out of pieces of paper. They just look like sand castles.

Historically, acquiring case law data has been a significant challenge, acting as a barrier to newcomers in the legal research market. Established players are often protective of their data. For instance, in an antitrust counterclaim, ROSS Intelligence accused Thomson Reuters of withholding their public law collection, claiming they had to instead resort to purchasing cases piecemeal from sources like Casemaker and Fastcase.  Other companies have taken more extreme measures. For example, Ravel Law partnered with the Harvard Law Library to scan every single opinion in their print reporter collections. There’s also speculation that major vendors might even license some of their materials directly to platforms like Google Scholar, albeit with stringent conditions.

The New Entrants

Despite the historic challenges, several new products have recently emerged offering advanced legal research capabilities:

  • Descrybe.ai (founded 2023) – This platform leverages generative AI to read and summarize judicial opinions, streamlining the search process. Currently hosting around 1.6 million summarized opinions, it’s available for free.
  • Midpage (2022) – Emphasizing the integration of legal research into the writing process, users can employ generative AI to draft documents from selected source (see Nicola Shaver’s short writeup on Midpage here). Midpage is currently free at app.midpage.ai.
  • CoPilot (by LawDroid, founded 2016) – Initially known for creating chatbots, LawDroid introduced CoPilot, a GPT-powered AI legal assistant, in 2023. It offers various tasks, including research, translating, and summarizing. CoPilot is available in beta as a web app and a Chrome extension, and is free for faculty and students.
  • Paxton.ai (2023) – Another generative AI legal assistant, Paxton.ai allows users to conduct legal research, draft documents, and more. Limited free access is available without signup at app.paxton.ai, although case law research will require you to sign up for a free account.
  • Alexi (2017) – Originally focused on Canadian law, Alexi provides legal research memos. They’ve recently unveiled their instant memos, powered by generative AI. Alexi is available at alexi.com and provides a free pilot.

Caselaw Access Project and Free Law Project

With the Caselaw Access Project, launched in 2015, Ravel Law and Harvard Law Library changed the game. Through their scanning project, Harvard received rights to the case law data, and Ravel gained an exclusive commercial license for 8 years. (When Lexis acquired Ravel a few years later, they committed to completing the project.) Although the official launch date of free access is February 2024, we are already seeing a free API at Ravel Law (as reported by Sarah Glassmeyer).

Caselaw Access Project data is only current through 2020 (scanning was completed in 2018, and has been supplemented by Fastcase donations through 2020) and does not include digital-first opinions. However, this gap is mostly filled through CourtListener, which contains a quite complete set of state and federal appellate opinions for recent years, painstakingly built through their network of web scrapers and direct publishing agreements. CourtListener offers an API (along with other options for bulk data use).

And indeed, Caselaw Access Project and Free Law Project just recently announced a dataset called Collaborative Open Legal Data (COLD) – Cases. COLD Cases is a dataset of 8.3 million United States legal decisions with text and metadata, suitable for use in machine learning and natural language processing projects.

Most of the legal research products I mentioned above do not disclose their precise source of their case law data. However, both Descrybe.ai and Midpage point to CourtListener as a partner. My theory/opinion is that many of the others may be using this data as well, and that these new, more reliable and more complete sources of data are responsible for fueling some amazing innovation in the legal research sphere.

What Holes Remain?

Reviewing the coverage of CourtListener and Caselaw Access Project it appears to me that they have, when combined:

  • 100% of all published U.S. case law from 2018 and earlier (state and federal)
  • 100% of all U.S. Supreme Court, U.S. Circuit Court of Appeals, and state appellate court cases

There are, nevertheless, still a few holes that remain in the coverage:

  • Newer Reporter Citations. Newer appellate court decisions may not have reporter citations within CourtListener. These may be supplemented as Fastcase donates cases to Caselaw Access Project.
  • Newer Federal District Court Opinions. Although CourtListener collects federal decisions marked as “opinions” within PACER, these decisions are not yet available in their opinion search. Therefore, very few federal district court cases are available for the past 3-4 years. This functionality will likely be added, but even when it is, district courts are inconsistent about marking decisions as “opinions” and so not all federal district court opinions will make their way to CourtListener’s opinions database. To me, this brings into sharp relief the failure of federal courts to comply with the 2002 E-Government Act, which requires federal courts to provide online access to all written opinions.
  • State Trial Court Decisions. Some other legal research providers include state court trial-level decisions. These are generally not published on freely available websites (so CourtListener cannot scrape them) and are also typically not published in print reporters (so Caselaw Access Project could not scan them).
  • Tribal Law. Even the major vendors have patchy access to tribal law, and CourtListener has holes here as well.

The Elephant in the Room

Of course, another major factor in the increase in legal research tools may be simple economics. In August, Thomson Reuters acquired the legal research provider Casetext for the eye-watering sum of $650 million.  And Casetext itself is a newer legal research provider, founded only in 2013. In interviews, Thomson Reuters cited Casetext’s access to domain-specific legal authority, as well as its early access to GPT-4, as key to its success. 

What’s Next?

Both Courtlistener and Caselaw Acess Project have big plans for continuing to increase access to case law. CAP will launch free API access in February 2024, coordinating with LexisNexis, Fastcase, and the Free Law Project on the launch. CourtListener is planning a scanning project to fix remaining gaps in their coverage (CourtListener’s Mike Lissner tells me they are interested in speaking to law librarians about this – please reach out). And I’m sure we can expect to see additional legal research tools, and potentially entire LLMs (hopefully open source!), trained on this legal data.

Know of anything else I didn’t discuss? Let me know in the comments, or find me on social media or email.

Keeping Up With Generative AI in the Law

Posted on by
5

The pace of generative AI development (and hype) over the past year has been intense, and difficult even for us experienced librarians, masters of information that we are, to follow. Not only is there a constant stream of new products, but also new academic papers, blog posts, newsletters, and more, from people evaluating, experimenting with, and critiquing those products. With that in mind, I’m sharing my favorites, and I’ll also pepper in a few recommendations from my co-bloggers.

Twitter

Before Twitter began its slow decline, it was one of my primary sources for professional connection, and there are many there who are exploring generative AI. I especially enjoy following people outside of the legal world. Many of my favorites are still there, like Ethan Mollick, Anna Mills, and Lance Eaton (all in higher education) as well as critical AI theorists like Timnit Gibru and Emily Bender.

LinkedIn

Despite the good bits that remain on Twitter, many interesting legal tech discussions seem to have moved to LinkedIn (or perhaps I’ve only recently found them there). Some of my favorites to follow on LinkedIn (in no particular order beyond how I’m running across them as I scroll) are: Nicole Black, Sam Harden, Alex Smith, Cat Moon, Damien Riehl, Dennis Kennedy, Uwais Iqbal, Ivy Grey, Robert Ambrogi, Cat Casey, Nicola Shaver, Adam Ziegler, and Michael Bommarito. Both Bob Ambrogi and Nicola Shaver recently had posts gathering legal tech luminaries to follow, so I would recommend checking out those posts and the comments to find more interesting folks. And if anyone else has figured out the LinkedIn etiquette for connecting vs. following someone you only know via other social media, please let me know.

Newsletters

Most of us have many (many, many) newsletters filling our inbox each day. Here are some favorites.

Jenny:

  • AI in Education – a Google group
  • Lawyer Ex Machina – from law librarian Eli Edwards, on legal technology, law practice and selected issues around big data, artificial intelligence, blockchain, social media and more affecting both the substance and the business of law (weekly)
  • The Neuron – AI news, tools, and how-to
  • The Brainyacts – from Josh Kubicki, insight & tips on generative AI use in legal services (daily)

Rebecca:

  • One Useful Thing – from Ethan Mollick, mostly on AI in higher ed (weekly)
  • Do Something – from Sam Harden, on legal tech, often from a small firm and access to justice angle
  • Legal Tech Trends – legal tech links, podcast, articles, products, along with original pieces (every two weeks or so)
  • KnowItAALL – this daily newsletters is a benefit for members of AALL (American Association of Law Libraries), but it is also available to non-members for a fee; great coverage of legal AI, I read it every day
  • AI Law Librarians – is it gauche to recommend our own blog? You can subscribe as a newsletter if you like!

Sean:

Podcasts

There are loads of podcasts on AI, but here are a few we follow:

Blogs & Websites

We’re bloggers, we like blogs. Traditional media can be ok, too, although mind the paywall.

YouTube

Sean also mentioned that much of the interesting stuff is on YouTube, but that it is fairly high-effort because many of the videos are an hour long, or more. Maybe we’ll convince him to share some of his favorite videos soon in a future post!

A Few LibGuides

If you still need more, here are a few libguides:

What about you?

Who are your favorites to follow on social media? Are there helpful newsletters, blogs, podcasts, or anything else that we’ve missed? Let us know in the comments.