Nimit Kalra

Verdict: A Library for Compound LLM Judge Systems
Nimit Kalra, Leonard Tang
[arXiv]   [code]   [docs]

Open-source library for scaling inference-time compute of LLM-as-a-judge systems by constructing arbitrary reasoning trace shapes. We achieve SOTA or near-SOTA performance on a wide variety of challenging automated evaluation tasks with no additional training.

Constitutional Classifiers: Defending against Universal Jailbreaks...
Mrinank Sharma, … Nimit Kalra, … Anthropic Safeguards Research Team
[arXiv]   [release]

We introduce Constitutional Classifiers, a framework that trains classifier safeguards using explicit constitutional rules. Our output classifiers support streaming prediction: they assess the potential harmfulness of the complete model output at each token without requiring the full output to be generated.

Domain Adaptation Through Task Distillation
ECCV 2020
Brady Zhou*, Nimit Kalra*, Philipp Krähenbühl
[arXiv]   [code]   [presentation]

Domain adaptation framework for transferring tasks between visually-diverse domains. We successfully transfer agents that navigate mazes and race karts to drive autonomously in a photorealistic simulator.

A Question for Cory Booker (D-NJ) — June 2022

I had the opportunity to ask — "Some of the most beautiful and compelling ideas that we cherish in our nation today were at one point quite controversial and unpopular. As you mentioned, the Civil Rights movement was not a result of Senators in suits deciding to grant rights — it was the hard work of passionate citizens who convinced a nation with their ideas. And yet, there was a time when a majority would’ve surely preferred to censor such ideas. What do you see as the future of social media regulation and how can we protect free debate online?"

21st Century Guard Labor — April 2022

It is fitting that being a "webmaster" requires no special certification or inclusion in some privileged class. Any script-kiddie from around the world can give a blog post the appearance of credibility by spoofing a publish date or changing the look-and-feel to match an academic article. Perhaps this is why the establishment and digital world are always at odds with another.