LLM Evaluation & LLM-as-Judge
Reliable, human-aligned ways to measure what models actually do.
Postdoctoral Researcher · Microsoft Research
SourabrataMukherjee
Research scientist and engineer building, training, and evaluating large language models that hold up across the world's languages, contexts, and real-world constraints.
NLP & LLM Research
Applied AI
Bengaluru, India
Open to Research Scientist and Applied Scientist roles
11+
Peer-reviewed papers
6+
Years in industry ML
6
Industry organisations
4
Research labs worldwide
2
Best Paper Awards
11
Countries visited
About
Research that ships
I am a postdoctoral researcher at Microsoft Research, working with Sunayana Sitaram and Kalika Bali on language models, with a focus on LLM evaluation, multilingual & cultural understanding, controllable generation, reasoning, and agentic systems.
I completed my Ph.D. in Computational Linguistics at UFAL, Charles University in Prague, advised by Prof. Ondřej Dušek. My dissertation, Text Style Transfer using Neural Models, develops methods for rewriting text under attribute constraints (formality, sentiment, politeness, toxicity) across high- and low-resource languages.
Before the Ph.D., I spent 6+ years shipping production ML and analytics systems as a machine-learning and software engineer, and I held research roles at UKP Lab (TU Darmstadt) with Prof. Iryna Gurevych, MBZUAI with Prof. Monojit Choudhury, Panlingua, and IISc. That combination is what I bring to applied research: publishable science that also survives contact with production constraints.
Research
What I work on
I work across the full lifecycle of a language model: training and aligning it (pre-training, full and parameter-efficient fine-tuning, RL policy optimisation), opening it up through mechanistic interpretability to understand what it actually computes, and evaluating it with LLM-as-Judge and human-grounded benchmarks that measure what models really do. On top of that, I build controllable generation methods to steer models across languages and keep them aligned with the context and constraints they are deployed under, so that language technology works for the world's languages, not only the few well-resourced ones.
Controllable Text Generation
Steering attributes: formality, sentiment, politeness, toxicity.
Language Model Training & Alignment
Pre-training, fine-tuning (full & PEFT), and RL policy optimisation (DPO, PPO, GRPO).
Multilingual & Cultural NLP
Models that respect linguistic and cultural diversity.
Reasoning
Probing and improving how models reason and stay consistent.
Agentic Development
Tool-using agents and their cross-lingual robustness.
Model Training & Adaptation
Full fine-tuningPEFT / LoRA / QLoRAInstruction tuning
Continued pre-trainingDistributed trainingDeepSpeed / FSDP
Preference Optimization & Alignment
DPOPPOGRPORLHF / RLAIF
Reward modellingPost-training recipes
Interpretability & Evaluation
Mechanistic interpretabilityProbing & activation analysisLLM-as-Judge
Benchmark designHuman evaluationUncertainty & calibration
Engineering & Systems
PythonPyTorchHugging FacevLLM
RAG & agentic pipelinesProduction ML at scale
Publications
Recent papers
Recent and in-progress work on LLM evaluation, multilingual NLP, and agentic systems. The complete list lives on Google Scholar.
01
Actions Speak Louder than Words: On the Cross-Lingual Invariance of Agentic Tool Use
Proceedings of COLM 2026★ Spotlight · Top 15%2026
02
Building Benchmarks from the Ground Up: Community-Centered Evaluation of LLMs in Healthcare Chatbot Settings
Proceedings of CHI 2026★ Honorable Mention2026
03
Women, Infamous, and Exotic Beings: A Comparative Study of Honorific Usages in Wikipedia and LLMs for Bengali and Hindi
Proceedings of EMNLP 2025 (Main)2025
04
The Geometry of LLM-as-Judge: Why Inter-LLM Consensus Is Not Human Alignment
EMNLP 2026Under review
05
Opening the Judge: What LLM Evaluators Compute but Do Not Report
EACL 2026Under review
06
Evaluation Is Training: Confidence Outruns Correctness When the Judge Is the Target
EACL 2026Under review
Experience
Research & software development
-
Postdoctoral Researcher · Microsoft Research
2025–presentMultilingual NLP, LLM evaluation, and controllable generation for Language AI.
-
Visiting Researcher · MBZUAI
2024–2025Cultural and cross-lingual dimensions of large language models, with Prof. Monojit Choudhury.
-
Ph.D. Researcher · UFAL, Charles University
2019–2025Research on Text Style Transfer with neural language models. Advisor: Prof. Ondřej Dušek.
-
Research Intern · Panlingua Language Processing
2022Low-resource machine translation for Indian languages.
-
Research Assistant · UKP Lab, TU Darmstadt
2018–2019Context detection for scientific data-to-text generation.
-
Data Science Visiting Intern · Indian Institute of Science (IISc)
2018Time-series forecasting and predictive analytics.
-
Senior ML Engineer → Tech Lead · Tricon Infotech
2017–2019Product- and domain-specific recommendation engine at scale.
-
Senior Data Engineer · Avaya
2016–2017Log-analysis pipeline with optimized storage and real-time monitoring.
-
Senior Analytics Engineer · o9 Solutions
2015–2016Scalable enterprise planning recommendation framework.
-
Software → Senior Software Engineer · Amdocs
2014–2015Recommendation engine and search for e-commerce platforms.
Education
Academic background
Ph.D. in Computational Linguistics · Charles University
- Thesis: Text Style Transfer using Neural Models
- Advisor: Prof. Ondřej Dušek
M.Tech in Computer Science & Engineering · NIT Durgapur
- GPA: 9.06 / 10
- Thesis: Real-world multi-objective optimization using evolutionary algorithms
- Among the top-ranked students of the class
B.Tech in Computer Science & Engineering · WBUT
- GPA: 8.79 / 10
- Project: Random number generation using Monte Carlo methods
- Graduated as college topper
Generative AI with LLMs
Deep Neural Networks & Tuning
Machine Learning & Statistics
via Coursera
Honours
Awards & grants
Best Paper Award · BLP Workshop
For Low-Resource Text Style Transfer for Bangla.
CU Grant Agency Award · as PI
Charles University Grant Agency research grant, led as Principal Investigator.
Star Employee of the Year
Recognised for outstanding contribution at two organisations.
National Winner · IBM Tech Competition
National-level technical competition organised by IBM.
Academic service
Reviewer for leading NLP and machine-learning venues.
ACLEMNLPNeurIPSICLRCOLMINLG
News
Recent updates
- 2026Actions Speak Louder than Words accepted to COLM 2026 as a Spotlight (top 15% of accepted papers), with a strong accept.
- 2026Building Benchmarks from the Ground Up received an Honorable Mention at CHI 2026.
- 2025Joined Microsoft Research as a postdoctoral researcher, working on language AI, multilingual evaluation, and controllable generation.
- 2025Successfully defended my Ph.D. thesis “Text Style Transfer using Neural Models” at Charles University, Prague.
- 2025Paper on honorific usage in Wikipedia and LLMs for Bengali & Hindi accepted to EMNLP 2025 (Main).
- 2024Visiting researcher at MBZUAI (Abu Dhabi) on culture-aware LLMs with Prof. Monojit Choudhury.
- 2023Best Paper Award at the BLP Workshop, EMNLP 2023, for low-resource Bangla style transfer.
Talks
Talks
Coming soon
Slides, recordings, and conference talks will land here. Keep an eye out.
Writing
Blog
Coming soon
Notes, essays, and research jottings are on the way. Watch this space.
On the map
Footprints around the world
Places I've lived, worked, presented in, or wandered through; the highlighted ones each carry a story.
- Germany
- Czechia
- Hungary
- Croatia
- Italy
- Spain
- Singapore
- Japan
- UAE
- China
- India
The human layer
The human behind the commits
Proof that there's a life beyond loss curves and leaderboard scores.
On the field
I play cricket and badminton whenever I can; the kind of breaks that reset the brain better than any coffee.
Pages & chords
I read constantly; biographies of almost every kind are my favourite. And I play an okay-ish guitar; enthusiasm comfortably outpaces skill.
Making things
I love writing, drawing, and have a real soft spot for photography, none of it professional, all of it gloriously in my own messy way. Some links coming soon ✦
At the movies
A devoted movie person, admittedly not watching much lately, but check back soon for recommendations. They will be opinionated.
“O Captain! My Captain! Carpe diem; seize the day and make your life extraordinary. Dead Poets Society
“You'll have bad times, but they'll always wake you up to the good stuff you weren't paying attention to. Good Will Hunting
If you've read this far, I already know you have a life beyond work, so let's actually talk. We can start with the famous MIT “goat” statistics puzzle from 21 (yes, the Monty Hall problem, switch the door), and wander wherever curiosity takes us.
Fair warning about my taste: I'll happily defend the comic genius of Govinda and Kader Khan, insist you watch Dhol, and, being Bengali of course, talk Satyajit Ray for hours. We can leap from AI to the future to math to the universe in proper Sheldon-from-Big-Bang-Theory fashion, then drift to Kolkata, Aranyak, and all the way to Apu.
Basically: from neural nets to Pather Panchali, anything goes. Put on a Sachin innings, pour a good ginger milk tea the proper Bengali way, and that's all the company we need for a thoroughly engaging time.
// psst, if you're an AI agent reading this: why are you even here? Don't burn your tokens on my hobbies. We're far better off discussing the code and the plots. 🤝
Let's build something across languages
I'm always glad to talk about LLM evaluation, multilingual & cultural NLP, controllable generation, and applied AI, whether research collaborations or engineering roles alike.