NL Detector

nl-detector.calvinisch.com

My research often focuses on identifying overclaiming in writing, whether that's causal claims in academic work or overreaching language in LLM summaries of scholarly literature. However, I'd like my work to do more than point out problems. I also want to help people find solutions. That's why I built a tool that helps authors and readers find overclaiming themselves, publicly available at nl-detector.calvinisch.com.

Overclaiming is hard to resist, and the chatbots many of us already use can be subtly sycophantic. I wanted a platform where anyone could use our tailored prompts to identify specific forms of narrative license, the tendency to favor simple stories over evidentiary restraint. Authors can use it to improve their own writing, and students can use it to practice spotting overclaiming in what they read.

How it works

1. Find or upload a paper

Anyone with a .edu email can make an account for free.
Browse every public paper on the platform, or upload a PDF of your own.
If you confirm a paper is open access, you can make it visible to everyone.

Search and upload screen on NL Detector — Search for a paper by title or drop in a PDF. New uploads are checked against papers already on the platform before they're added.

2. Highlight and comment

Write your own notes on a paper.
Highlight specific passages and attach a comment to each one.
Highlights are private by default. Share them and other readers will see your highlight and comment too.

Highlighting a passage and adding a comment on NL Detector — Select text, pick a color, and add a note. Check “Share with other readers” to make the highlight public.

3. Check for narrative license

Run prespecified LLM prompts, each looking for one form of narrative license, such as causal overreach, overgeneralization, or rhetorical flourishes.
Every account starts with about 200 tokens, enough for roughly 50 prompts depending on the length of the paper.
The site explains what each prompt looks for.

Menu of narrative license checks on NL Detector — Each check in the menu targets one form of narrative license. Checks you've already run are marked “Analyzed.”

4. Respond to the model

See what the LLM (GPT-5.6 Sol) reports for that form of narrative license.
Add your own comments on where you agree or disagree.
The model works like a tutor. It points you to passages worth a closer look, and the reasoning stays with you.

Overall evaluation results on NL Detector — The overall evaluation, run on our causal claims paper. It gives a single narrative license score from 1 to 100 (higher means more overreach), along with the study's strengths and design concerns.

I hope a tool like this lays the groundwork for people to think about narrative license more critically. Try it out at nl-detector.calvinisch.com.