Nimit Kalra
Hey there! I'm a researcher at Haize Labs, focusing on improving adversarial robustness and automated model-based evaluation for LLMs. I authored Verdict, a framework for specifying compound LLM judge systems. Before that, I spent three years at Citadel.
During my time at UT Austin, I was advised by Philipp Krähenbühl on domain adaptation in computer vision and robotics, with a particular emphasis on data efficiency. I graduated with a B.A. in Math, a B.S. in Computer Science, and a bunch of credits in Political Science and Economics.
Publications
|
Verdict: A Library for Compound LLM Judge Systems Open-source library for scaling inference-time compute of LLM-as-a-judge systems by constructing arbitrary reasoning trace shapes. We achieve SOTA or near-SOTA performance on a wide variety of challenging automated evaluation tasks with no additional training. |
|
|
Constitutional Classifiers: Defending against Universal Jailbreaks... We introduce Constitutional Classifiers, a framework that trains classifier safeguards using explicit constitutional rules. Our output classifiers support streaming prediction: they assess the potential harmfulness of the complete model output at each token without requiring the full output to be generated. |
|
|
Domain Adaptation Through Task Distillation Domain adaptation framework for transferring tasks between visually-diverse domains. We successfully transfer agents that navigate mazes and race karts to drive autonomously in a photorealistic simulator. |
Writing
|
Bootstrapping Supervision — June 2025 In the meantime, some paper summaries: Absolute Zero Reasoner (Zhao et al. May 2025) |
Projects
[report] Statement of Purpose for Computer Science Ph.D. Programs
[report] Domain Adaptation Through Multi-Task Distillation via Noisy-Labels
[report] A Bayesian Network Model for Sampling Dockless Scooter Traffic
[report] [code] Fast Random Kernelized Features: High-Dimensional SVM Classification
Adventures
I enjoy a good road trip.
People
People who have had a major impact on me — whether a sparring buddy, mentor, or friend.
Jagath ∙ Alex ∙ Srujay ∙ Leonard ∙ Philipp ∙ Brady ∙ Yuwei ∙ Saaketh ∙ Dylan ∙ Will ∙ Prateek
Contact
I love meeting new people. Reach me at nimit@utexas.edu or schedule a chat.