Legal · North Star Legal Partners
AI Contract-Review Agent for a Top-50 Law Firm
Built a multi-agent contract-review system that flags risk clauses, drafts redlines, and produces partner-ready summaries — with a rigorous evaluation harness the firm's GC team trusts.
Problem
Partners were spending 20+ hours per week on first-pass contract review. Off-the-shelf legal AI tools produced confident-sounding hallucinations and had no path to evaluate accuracy. The firm's IT team had explicitly banned generic ChatGPT usage over confidentiality concerns.
Solution
We built an in-VPC agentic system on Claude and GPT-5 with tool use, pgvector-backed retrieval over the firm's precedent library, and a Braintrust eval harness scoring 400+ labeled contract fragments per model release. Every generated redline is grounded in a citation the partner can click through.
Result
Agent now handles first-pass review on 78% of standard NDAs, SOWs, and MSAs. Partners reclaimed an average of 1,100 hours per quarter. Zero confidentiality incidents in production. Eval accuracy has stayed above 96% across three model swaps.