Research output
Publications
Selected work, grouped by primary research contribution. Full list on Google Scholar.
Efficient & Scalable Foundation Models
Architectures, pre-training, compression, and systems for capable models at lower cost.
- Diversifying the Expert Knowledge for Task-Agnostic Pruning in Sparse Mixture-of-Experts, ACL 2025
- Phi-3 Technical Report: A Highly Capable Language Model Locally on Your Phone (2024)
- Fast-ELECTRA for Efficient Pre-training, ICLR 2024
- Efficient Long Sequence Modeling via State Space Augmented Transformer, COLM 2024
- Bridging Discrete and Backpropagation: Straight-Through and Beyond, NeurIPS Oral 2023
- METRO: Efficient Denoising Pretraining of Large Scale Autoencoding Language Models (2022)
- DeBERTa: Decoding-Enhanced BERT with Disentangled Attention, ICLR 2021
- Domain-Specific Language Model Pretraining for Biomedical NLP, ACM TCHC 2021 Best Paper Award
- UniLMv2: Pseudo-Masked Language Models for Unified Language Model Pre-Training, ICML 2020
- Unified Language Model Pre-training for Natural Language Understanding and Generation, NeurIPS 2019
- Improving Multi-Task Deep Neural Networks via Knowledge Distillation for Natural Language Understanding (2019)
- Multi-Task Deep Neural Networks for Natural Language Understanding, ACL 2019 — First to outperform BERT on GLUE
Post-training, RL & Reasoning
Adaptation, distillation, reinforcement learning, and efficient reasoning.
- Weak-to-Strong On-Policy Distillation (2026)
- GFlowRL: Scaling Distribution-Matching RL to Large Language Models (2026)
- Training Large Reasoning Models Efficiently via Progressive Thought Encoding, ICLR 2026
- FlowRL: Matching Reward Distributions for LLM Reasoning, ICLR 2026
- Multimodal Reinforcement Learning with Adaptive Verifier for AI Agents (2025)
- Generative Adapter: Contextualizing Language Models in Parameters with A Single Forward Pass, ICLR 2025
AI Safety, Robustness & Responsible AI
Safety evaluation, adversarial robustness, red teaming, and high-stakes deployment.
- Evaluating the Robustness and Readiness of Large Frontier Models in Health AI Applications, Nature Medicine 2026
- Jailbreak Susceptibility Prediction and Mitigation via the Behavioral Geometry of Models (2026)
- MultiBreak: A Scalable and Diverse Multi-turn Jailbreak Benchmark for Evaluating LLM Safety, ICML 2026
- Statistical Estimation of Adversarial Risk in Large Language Models under Best-of-N Sampling (2026)
- SEMA: Simple yet Effective Learning for Multi-Turn Jailbreak Attacks, ICLR 2026
- Iterative Self-Tuning LLMs for Enhanced Jailbreaking Capabilities, NAACL Oral 2025
- Adversarial Training for Large Neural Language Models (2020)