No. 02of 22LandmarkResearch

Legal NLP research and a platform for Indian law

LawVerse

A legal research platform for Indian law, and the study behind it. Across 231,908 court judgments, our first pipeline said 22 of 25 legal concepts had changed in meaning. After we found and fixed four silent defects in it, one did.

When
Mar – Nov 2026
Status
Research
Team
RV University computer science capstone project, built by a team.
Role
Co-author of the paper. Listed as team lead and full-stack developer in the project README.

Progress

Progress75%

Research pipeline, citation graph, website, gateway and Android app all run locally. Not deployed publicly; the paper is under revision.

What comes next

  1. Revise and resubmit the paper.
  2. Add review cases that are real and verified by a lawyer, not synthetic.
  3. Deploy the platform behind HTTPS so the app can connect to it.

22 → 1

of 25 concepts read as changing, before and after the pipeline was corrected

The 22 is counted without the reliability floor, which the original pipeline can never pass; the 1 is counted with it. Without the floor, the corrected pipeline reads 7 of 25.

Workshop manuscript and result files, August 2026Verified. Backed by a file or public page that has been checked.Workshop manuscript and result files, August 2026 — The paper is not public: submitted to NLLP @ EMNLP 2026, not accepted, under revision.

231,908

court judgments in the corpus

99.7% Indian. The 800 US documents are too few for any cross-country comparison.

Workshop manuscript and result files, August 2026Verified. Backed by a file or public page that has been checked.Workshop manuscript and result files, August 2026

1 → 24

of 25 concepts flagged, from changing only which tokens are pooled

Workshop manuscript and result files, August 2026Verified. Backed by a file or public page that has been checked.Workshop manuscript and result files, August 2026

99.7% vs 23.7%

concept-identification accuracy: the corrected representation against [CLS] pooling

Nearest-centroid probe over 25 concepts. Chance is 4%.

Workshop manuscript and result files, August 2026Verified. Backed by a file or public page that has been checked.Workshop manuscript and result files, August 2026

98.8%

of 169 reliable decade-to-decade transitions fall in the lowest drift band

Workshop manuscript and result files, August 2026Verified. Backed by a file or public page that has been checked.Workshop manuscript and result files, August 2026

115,537

cases in the citation graph, joined by 67,864 citation links

Built from the indexed corpus only. How a later court treated a case is inferred from cue phrases near the citation.

Citation build log (build_citations.log)Verified. Backed by a file or public page that has been checked.Citation build log (build_citations.log) — Read in the project audit of 13 November 2026.

Exhibit 01 · Live demo

Live demo

This interactive shows how many of 25 legal concepts read as undergoing major semantic change, meaning an overall drift above 0.1, under different implementations of the same measure. Under the natural implementation,22 of 25 do. Under the corrected implementation, with a reliability floor, 1of 25 does; without the floor, 7 of 25. A table of every number follows the charts.

22of 25 concepts read as undergoing major change

Natural implementation · without the reliability floor

With the floor none can be scored: this variant keeps one pre-averaged vector per decade, so no decade reaches 5 documents.

Implementation
Corrected, with one choice changed
  • one concept, band only: the file records how many concepts fall in each band, not which
  • one concept, measured drift with its 95% bootstrap interval
  • 0.1: above this line a concept leaves the lowest band

Natural implementation

All four defects together: a window from the start of the judgment, bag-of-token matching, a bare-phrase fallback for unmatched decades, and one pre-averaged vector per decade.

19.6% of its decade cells share a vector with another decade of the same concept, and 100% of its confidence intervals have zero width. Its largest drift is 1.0106.

Aliased decade cells
19.6%

59 of 301 cells share a vector

Median transition drift
0.0360

all transitions

Largest transition drift
1.0106

all transitions

Rank agreement with corrected
ρ 0.42

over 266 shared transitions

Zero-width intervals
100%

includes single-document decades, which cannot be resampled

Identifies its own concept
not probed

no probe result on file

Corrected build · overall drift per concept, with 95% interval

  1. due processMERGER0.1726
  2. arbitrationMERGER0.0850
  3. judicial reviewMERGER0.0757
  4. bailMERGER0.0687
  5. consumer protectionMERGER0.0660
  6. probable causeMERGER0.0615
  7. seditionMERGER0.0536
  8. equal protectionBIFURCATION0.0487
  9. dowryMERGER0.0446
  10. negligenceMERGER0.0433
  11. fiduciary dutyMERGER0.0419
  12. fair trialEXPANSION0.0396
  13. environmental lawSTABLE0.0387
  14. privacyMERGER0.0356
  15. right to educationSTABLE0.0339
  16. reasonable doubtBIFURCATION0.0333
  17. free speechMERGER0.0326
  18. search and seizureSTABLE0.0307
  19. right to lifeEXPANSION0.0291
  20. freedom of expressionEXPANSION0.0274
  21. breach of contractEXPANSION0.0265
  22. contempt of courtSTABLE0.0257
  23. habeas corpusSTABLE0.0187
  24. defamationMERGER0.0126
  25. cyber crimeMERGER0.0080

The stored pre-fix build, made earlier on a different corpus revision, put 20 of 25 concepts outside the lowest band. Measured correctly, 1 of 25 is. Band names are the taxonomy the project inherited; the paper argues a single cosine distance cannot support them and reports magnitude only.

due process

pre-fix build MERGER 0.7 and above, then corrected EXPANSION 0.1 to 0.3

Overall drift, 1940s to 2020s
0.1726
95% bootstrap interval
0.1260 to 0.2574
Percentile among reliable transitions
99.4
Documents embedded
1,294
Decades: reliable of matched
9 of 16
Judgments containing the phrase
2,803

Drift by decade · 1 of 15 transitions on file

DecadesDrift95% intervalDocuments
2010s → 2020s0.18020.1643 to 0.1963198 → 200

The result files keep the 15 highest and 5 lowest of the build’s 169 reliable transitions, not every concept’s full timeline.

The corpus · 231,908 judgments by decade

The 1990s hold 22.5% of the corpus; no decade before 1940 holds more than 0.2%. Years run 1800 to 2029 as recorded; the paper attributes 317 documents (0.14%) to date-extraction errors.

By jurisdiction

India
231,108 99.7%
US
800 0.3%
  • One jurisdiction, in effect. The paper makes no cross-jurisdictional comparison.
  • No retrieval benchmark. No annotated query set exists.
  • No ground truth for drift. No expert-annotated set of doctrinal shifts, so no accuracy is reported.
Every number as a table
Concepts outside the lowest band, by implementation variant (ablation.json, probe.json)
VariantWithout floorWith floorAliased cellsMedian drift, no floorLargest drift, no floorρ vs correctedZero-width intervalsConcept identification
Natural implementation22 of 25none scored19.6%0.03601.01060.42100%not probed
Corrected7 of 251 of 250.0%0.01980.4203–22%99.7% (25 concepts)
F1 · window from char 011 of 233 of 200.0%0.03580.22130.5126%99.1% (15 concepts)
F2 · bag-of-token match7 of 251 of 250.0%0.02010.40290.9823%99.2% (25 concepts)
F3 · bare-phrase fallback8 of 251 of 253.0%0.01780.66651.0025%not probed
No per-document L2 norm7 of 251 of 250.0%0.01940.42031.0022%not probed
Mean-pool all tokens1 of 250 of 250.0%0.01070.18360.9421%42.6% (25 concepts)
[CLS] token24 of 2518 of 250.0%0.10160.80490.9221%23.7% (25 concepts)
Corrected build, per concept (paper_stats.json)
ConceptDecades comparedOverall drift95% intervalPercentileDocumentsReliable / matched decadesPre-fix build labelCorrected label
due process1940s to 2020s0.17260.1260 to 0.257499.41,2949 / 16MERGEREXPANSION
arbitration1920s to 2020s0.08500.0553 to 0.139798.81,65511 / 18MERGERSTABLE
judicial review1850s to 2020s0.07570.0621 to 0.095098.81,41211 / 17MERGERSTABLE
bail1820s to 2020s0.06870.0590 to 0.084698.81,32212 / 23MERGERSTABLE
consumer protection1970s to 2020s0.06600.0372 to 0.107798.87756 / 11MERGERSTABLE
probable cause1950s to 2020s0.06150.0444 to 0.082298.23128 / 9MERGERSTABLE
sedition1940s to 2020s0.05360.0384 to 0.069897.61459 / 9MERGERSTABLE
equal protection1940s to 2020s0.04870.0365 to 0.066096.41,4819 / 13BIFURCATIONSTABLE
dowry1860s to 2020s0.04460.0258 to 0.069194.11,1139 / 11MERGERSTABLE
negligence1860s to 2020s0.04330.0339 to 0.052593.51,63111 / 22MERGERSTABLE
fiduciary duty1970s to 2020s0.04190.0319 to 0.053292.91086 / 8MERGERSTABLE
fair trial1950s to 2020s0.03960.0346 to 0.045092.31,2938 / 12EXPANSIONSTABLE
environmental law1980s to 2020s0.03870.0272 to 0.056192.31135 / 5STABLESTABLE
privacy1950s to 2020s0.03560.0278 to 0.044691.18438 / 11MERGERSTABLE
right to education1980s to 2020s0.03390.0250 to 0.044990.51695 / 8STABLESTABLE
reasonable doubt1860s to 2020s0.03330.0253 to 0.043489.91,59910 / 14BIFURCATIONSTABLE
free speech1950s to 2020s0.03260.0244 to 0.040789.33048 / 9MERGERSTABLE
search and seizure1950s to 2020s0.03070.0265 to 0.034988.81,1278 / 8STABLESTABLE
right to life1950s to 2020s0.02910.0221 to 0.038088.89117 / 9EXPANSIONSTABLE
freedom of expression1950s to 2020s0.02740.0219 to 0.033387.62398 / 10EXPANSIONSTABLE
breach of contract1950s to 2020s0.02650.0235 to 0.029487.61,5358 / 10EXPANSIONSTABLE
contempt of court1950s to 2020s0.02570.0210 to 0.032185.83348 / 10STABLESTABLE
habeas corpus1950s to 2020s0.01870.0164 to 0.021777.51,5538 / 13STABLESTABLE
defamation1950s to 2020s0.01260.0100 to 0.016067.59198 / 13MERGERSTABLE
cyber crime2010s to 2020s0.00800.0050 to 0.013549.7882 / 3MERGERSTABLE
Bands inherited from earlier practice (paper, section 5)
BandOverall drift
STABLEbelow 0.1
EXPANSION0.1 to 0.3
CONTRACTION0.3 to 0.5
BIFURCATION0.5 to 0.7
MERGER0.7 and above
Corpus by decade, jurisdiction and source (paper_stats.json)
GroupDocumentsShare
1800s540.02%
1810s550.02%
1820s570.02%
1830s770.03%
1840s640.03%
1850s1460.06%
1860s3950.17%
1870s760.03%
1880s1200.05%
1890s1660.07%
1900s1460.06%
1910s1010.04%
1920s1590.07%
1930s1520.07%
1940s5990.26%
1950s13,1525.67%
1960s21,2919.18%
1970s22,2479.59%
1980s29,07512.54%
1990s52,10322.47%
2000s43,99818.97%
2010s28,48412.28%
2020s19,1918.28%
Jurisdiction: India231,10899.66%
Jurisdiction: US8000.34%
Source: Indian Kanoon (scraped)115,53049.82%
Source: HuggingFace Indian High Courts47,62320.54%
Source: HuggingFace Indian Supreme Court42,66818.40%
Source: HuggingFace Indian (other)25,28710.90%
Source: CourtListener (US federal)8000.34%

Exhibit 02 · Compare

One estimator, three verdicts

Original pipeline

22 of 25

concepts read as changing. Four silent defects were at work, and none raised an error. Counted without the reliability floor, which this pipeline can never pass.

Corrected pipeline

1 of 25

Window centered on the mention, exact phrase matching, no fallback, one vector per document. Counted with the reliability floor; without it, the corrected pipeline reads 7 of 25.

Corrected, [CLS] pooling

24 of 25

Nothing changed except which token positions are pooled. This representation identifies the right concept 23.7% of the time, against 99.7% for the corrected one. Chance is 4%.

From the paper's ablation and probe result files. Same data and same concepts in every state.

Exhibit 03 · Gallery

The Android app, GLaw

Exhibit 04 · Scorecard

Document review: the best run of 60 cases

The test cases are synthetic, and this best run is still just below the project's own release gate (at least 98% of serious defects caught). A later run the same day had provider failures, with no model response on 48 of 60 cases, so it is not counted as an accuracy figure.

Document review: the best run of 60 cases
MeasureResultNoteSource
Document type named correctly98.4%Synthetic cases.Evaluation run of 10 November 2026
Serious defects caught95.8%Below the release gate of 98%. Not passed.Evaluation run of 10 November 2026
Quoted passages that are valid97.0%Each finding must quote the text it rests on.Evaluation run of 10 November 2026
Redaction leaks0Identity numbers such as Aadhaar must stay redacted in the output.Evaluation run of 10 November 2026
Cost of a 10-page scan₹1.69Evaluation run of 10 November 2026
Median time for a review9.8 sEvaluation run of 10 November 2026

01

What it is

The Semantic Movement Index, SMI, is a simple estimator. Embed every mention of a legal phrase such as "due process" or "bail", average the vectors within a decade, and take one minus the cosine similarity between two decades. We ran it over 231,908 judgments, 99.7% of them Indian, with a legal-domain encoder.

LawVerse is the platform built around it: search over Indian judgments by meaning, a citation graph that shows how later courts treated a case, doctrine timelines for 25 legal concepts, and an AI review of contracts and property documents. Its Android app is called GLaw.

02

The hard part

The first build produced a striking result, complete with confidence intervals, and it was wrong. Four defects were responsible, and none of them raised an error or a warning. The text window was cut from the start of each judgment, so it usually did not contain the concept at all. The matcher matched loose words instead of the phrase. A fallback gave every unmatched decade the same vector. And the bootstrap resampled a single row, so every interval had zero width by construction. With no annotated benchmark for legal concept drift, nothing outside the pipeline could tell us it was broken.

03

What was built

A corrected pipeline: the window centered on the mention, contiguous phrase matching, no fallback, and one vector kept per document. Build-time checks that fail when two periods share a vector or an interval has zero width. A controlled ablation that changes one design choice at a time on identical data. A cheap probe that asks whether a representation can tell which concept it came from.

Around it, the platform. A cleaned corpus and a citation database of 115,537 cases joined by 67,864 citation links, with a label for how later courts treated each case. Semantic search. A document review in which a language model reads contracts and Indian property documents in 12 languages, quotes the passage behind each finding and redacts Aadhaar numbers. A FastAPI gateway, a Next.js website, an admin dashboard, and a Flutter Android app with a document scanner.

04

What was found

Rebuilt from the old code, the original pipeline labels 22 of 25 concepts as changing. The corrected one labels 1. The defects interact: the fallback alone corrupts 3.0% of cells, but 19.6% once the bad window is added. One defect turned out to be plainly wrong and yet harmless on everything we could measure, and we report it that way. The largest effect was not a bug at all. Changing only which token positions are pooled swings the verdict from 1 to 24 of 25, and the most dramatic drift comes from the representation that is worst at identifying a concept (23.7% against 99.7%). Measured correctly, 98.8% of the 169 reliable decade-to-decade transitions sit in the lowest band.

The document review has its own evaluation harness with a release gate. On its best run of 60 cases it named the document type correctly 98.4% of the time and caught 95.8% of serious defects. Those cases are synthetic, and 95.8% is just below the gate of 98%, so the review has not yet passed its own bar.

05

What it does not claim

The study is a negative result with a fix. Its paper, "When the Metric Measures the Bug", was submitted to NLLP @ EMNLP 2026, not accepted, and is under revision. It makes no cross-country claim, because only 800 documents are from the US and none are European. It reports no retrieval benchmark, because no annotated query set exists. The platform and the app run on local machines; nothing is deployed publicly. Earlier descriptions of this project, including its public README, carry figures from before the correction; this page is the current account. The next research steps are the ones the paper names: repeat the measurement with other encoders, and build an annotated benchmark for legal concept drift.

Figure

Histogram of all 169 reliable decade-to-decade SMI values on a log scale, with a median of 0.0080. All but two fall below the first band boundary at 0.1. The boundaries at 0.3, 0.5 and 0.7 are never reached.
The paper's figure. Measured correctly, 98.8% of transitions sit below the first band boundary.

One question

The answer is on this page. Get it right and it goes in your passport on the map.

Challenge

The first pipeline said 22 of 25 legal concepts had changed in meaning. After four silent defects were fixed, how many did?

Show the answer

1 of 25. 1 of 25. Rebuilt from the old code, the original pipeline flags 22; the corrected one flags 1, counted with its reliability floor. Source: the paper's ablation result file.

Stack
  • Python
  • PyTorch
  • Hugging Face Transformers
  • LexLM legal-RoBERTa-large
  • BGE-large-en-v1.5
  • FastAPI
  • SQLAlchemy and SQLite
  • Gemini (Groq as fallback)
  • Next.js
  • React
  • Flutter
  • ML Kit