Nimit Kalra

Verdict: A Library for Compound LLM Judge Systems
2025
Nimit Kalra, Leonard Tang
[arXiv]   [code]   [docs]

Open-source library for building compound LLM judge systems that enables building judges that achieve SOTA or near-SOTA performance on a wide variety of challenging automated evaluation tasks with no additional training.

Constitutional Classifiers: Defending against Universal Jailbreaks across Thousands of Hours of Red Teaming
2025
Mrinank Sharma, … Nimit Kalra, … Anthropic Safeguards Research Team

We introduce Constitutional Classifiers: safeguards trained on synthetic data, generated by prompting LLMs with natural language rules (i.e., a constitution) specifying permitted and restricted content.

Domain Adaptation Through Task Distillation
European Conference on Computer Vision, 2020
Brady Zhou*, Nimit Kalra*, Philipp Krähenbühl
[arXiv]   [code]   [presentation]

Domain adaptation framework for transferring tasks between visually-diverse domains. We successfully transfer agents that navigate mazes and race karts to drive autonomously in a photorealistic simulator.

A Question for Cory Booker (D-NJ) — June 2022

I had the opportunity to ask — "Some of the most beautiful and compelling ideas that we cherish in our nation today were at one point quite controversial and unpopular. As you mentioned, the Civil Rights movement was not a result of Senators in suits deciding to grant rights — it was the hard work of passionate citizens who convinced a nation with their ideas. And yet, there was a time when a majority would’ve surely preferred to censor such ideas. What do you see as the future of social media regulation and how can we protect free debate online?"

21st Century Guard Labor — April 2022

It is fitting that being a "webmaster" requires no special certification or inclusion in some privileged class. Any script-kiddie from around the world can give a blog post the appearance of credibility by spoofing a publish date or changing the look-and-feel to match an academic article. Perhaps this is why the establishment and digital world are always at odds with another.