Publications

Selected and recent work across efficient ML systems, LLM infrastructure, federated learning, optimization, and trustworthy machine learning.

32 papers 9 years 13 venues

Showing 32 publications

2025

arXiv 2025

K2-V2: A 360-Open, Reasoning-Enhanced LLM

K2 Team, arXiv 2025 [arXiv]

LLM Systems Open Models

2024

NeurIPS 2024

SHED: Shapley-Based Automated Dataset Refinement for Instruction Fine-Tuning

Y. He, Z. Wang, Z. Shen, G. Sun, Y. Dai, Y. Wu, H. Wang, A. Li, NeurIPS 2024 [arXiv]

LLM Systems Data & Evaluation

NeurIPS Datasets and Benchmarks Track 2024

Model-GLUE: Democratized LLM Scaling for A Large Model Zoo in the Wild

X. Zhao, G. Sun, R. Cai, Y. Zhou, P. Li, P. Wang, B. Tan, Y. He, L. Chen, Y. Liang, B. Chen, B. Yuan, H. Wang, A. Li, Z. Wang, T. Chen, NeurIPS 2024 Datasets and Benchmarks [link]

LLM Systems Open Models

NeurIPS 2024

FLoRA: Federated Fine-Tuning Large Language Models with Heterogeneous Low-Rank Adaptations

Z. Wang, Z. Shen, Y. He, G. Sun, H. Wang, L. Lyu, A. Li, NeurIPS 2024 [arXiv]

LLM Systems Federated Learning

ICML 2024

TrustLLM: Trustworthiness in Large Language Models

H. Wang with many collegues (Position Paper), ICML 2024 [link] [arXiv]

LLM Systems Data & Evaluation

ICML 2024

Maestro: Uncovering Low-Rank Structures via Trainable Decomposition

S. Horváth, S. Laskaridis, S. Rajput, H. Wang, ICML 2024 [link] [arXiv]

Optimization Model Compression

COLM 2024

LLM360: Towards Fully Transparent Open-Source LLMs

Z. Liu, A. Qiao, W. Neiswanger, H. Wang, B. Tan, T. Tao, J. Li, Y. Wang, S. Sun, O. Pangarkar, R. Fan, Y. Gu, V. Miller, Y. Zhuang, G. He, H. Li, F. Koto, L. Tang, N. Ranjan, Z. Shen, R. Iriondo, C. Mu, Z. Hu, M. Schulze, P. Nakov, T. Baldwin, E. P. Xing, COLM 2024 [arXiv]

LLM Systems Open Models

COLM 2024

Crystal: Illuminating LLM Abilities on Language and Code

T. Tao, J. Li, B. Tan, H. Wang, W. Marshall, B. M Kanakiya, J. Hestness, N. Vassilieva, Z. Shen, E. P. Xing, Z. Liu, COLM 2024 [arXiv]

LLM Systems Data & Evaluation

NAACL Demo 2024

RedCoast: A Lightweight Tool to Automate Distributed Training of LLMs on Any GPU/TPUs

B. Tan, Y. Zhu, L. Liu, H. Wang, Y. Zhuang, J. Chen, E. P. Xing, Z. Hu, NAACL Demo 2024 ($\color{red}{\text{the Best Demo Runner Up}}$) [link] [arXiv]

LLM Systems Distributed Training

MLSys 2024

Does compressing activations help model parallel training?

S. Bian, D. Li, H. Wang, E. P. Xing, S. Venkataraman, MLSys 2024 [arXiv]

Distributed Training Model Compression

ICLR 2024

Fusing Models with Complementary Expertise

H. Wang, F. M. Polo, Y. Sun, S. Kundu, E. P. Xing, M. Yurochkin, ICLR 2024 [link] [arXiv]

Optimization Model Fusion

2023

NeurIPS 2023

FedNAR: Federated Optimization with Normalized Annealing Regularization

J. Li, A. Li, C. Tian, Q. Ho, E. Xing, H. Wang, NeurIPS 2023 [link] [arXiv]

Federated Learning Optimization

MLSys 2023

Cuttlefish: Low-rank Model Training without All The Tuning

H. Wang, S. Agarwal, P. U-chupala, Y. Tanaka, E. P. Xing, D. Papailiopoulos, MLSys 2023 [link] [arXiv]

Efficient Training Optimization

ICLR 2023

MPCFormer: fast, performant and private Transformer inference with MPC

D. Li*, R. Shao*, H. Wang*, H. Guo, E. P. Xing, H. Zhang, ICLR 2023, ($\color{red}{\text{Spotlight}}$) [link]

Privacy & Security Efficient Inference

ICLR 2023

Federated Learning as Variational Inference: A Scalable Expectation Propagation Approach

H. Guo, P. Greengard, H. Wang, A. Gelman, E. P. Xing, Y. Kim, ICLR 2023 [link]

Federated Learning Optimization

2022

Findings of EMNLP 2022

Efficient Federated Learning on Knowledge Graphs via Privacy-preserving Relation Embedding Aggregation

K. Zhang, Y. Wang, H. Wang, L. Huang, C. Yang, X. Chen, L. Sun, Findings of EMNLP 2022

Federated Learning Privacy & Security

NeurIPS 2022

Rare Gems: Finding Lottery Tickets at Initialization

K. Sreenivasan, J. Sohn, L. Yang, M. Grinde, A. Nagle, H. Wang, E. P. Xing, K. Lee, D. Papailiopoulos, NeurIPS 2022 [arXiv]

Optimization Deep Learning

NeurIPS 2022

AMP: Automatically Finding Model Parallel Strategies with Heterogeneity Awareness

D. Li, H. Wang, E. P. Xing, H. Zhang, NeurIPS 2022 [arXiv]

Distributed Training Model Parallelism

MLSys 2022

On the Utility of Gradient Compression in Distributed Training Systems

S. Agarwal, H. Wang, S. Venkataraman, D. Papailiopoulos, MLSys 2022 [link] [arXiv]

Distributed Training Optimization

2021

MLSys 2021

Pufferfish: Communication-efficient Models At No Extra Cost

H. Wang, S. Agarwal, D. Papailiopoulos, MLSys 2021 [arXiv] [link] [talk]

Efficient Training Model Compression

MLSys 2021

Accordion: Adaptive Gradient Communication via Critical Learning Regime Identification

S. Agarwal, H. Wang, K. Lee, S. Venkataraman, D. Papailiopoulos, MLSys 2021, [arXiv] [link] [talk]

Distributed Training Optimization

2020

NeurIPS 2020 SpicyFL workshop 2020

FedML: A Research Library and Benchmark for Federated Machine Learning

C. He, S. Li, J. So, M. Zhang, H. Wang, X. Wang, P. Vepakomma, A. Singh, H. Qiu, L. Shen, P. Zhao, Y. Kang, Y. Liu, R. Raskar, Q. Yang, M. Annavaram, S. Avestimehr, NeurIPS 2020 SpicyFL workshop, ($\color{red}{\text{the Baidu Best Paper Award}}$) [arXiv]

Federated Learning ML Systems

NeurIPS 2020

Attack of the Tails: Yes, You Really Can Backdoor Federated Learning

H. Wang, K. Sreenivasan, S. Rajput, H. Vishwakarma, S. Agarwal, J. Sohn, K. Lee, D. Papailiopoulos, NeurIPS 2020, [link]

Federated Learning Privacy & Security

ICLR 2020

Federated Learning with Matched Averaging

H. Wang, M. Yurochkin, Y. Sun, D. Papailiopoulos, Y. Khazaeni, ICLR 2020, ($\color{red}{\text{Oral}}$) [link][blog][talk]

Federated Learning Model Fusion

2019

NeurIPS 2019

DETOX: A Redundancy-based Framework for Faster and More Robust Gradient Aggregation

S. Rajput*, H. Wang*, Z. Charles, D. Papailiopoulos, NeurIPS 2019, [link]

Distributed Training Robustness

ACM SIGMOD, demo track 2019

Demonstration of Nimbus: Model-based Pricing for Machine Learning in a Data Marketplace

L. Chen, H. Wang, L. Chen, P. Koutris, A. Kumar, ACM SIGMOD 2019 demo track, [link]

ML Systems Data Markets

arXiv 2019

ErasureHead: Distributed Gradient Descent without Delays Using Approximate Gradient Coding

H. Wang, Z. Charles, D. Papailiopoulos [arXiv]

Distributed Training Gradient Coding

2018

NeurIPS 2018

The Effect of Network Width on the Performance of Large-batch Training

L. Chen, H. Wang, J. Zhao, D. Papailiopoulos, P. Koutris, NeurIPS 2018, [link]

Optimization Deep Learning

NeurIPS 2018

ATOMO: Communication-efficient Learning via Atomic Sparsification

H. Wang*, S. Sievert*, Z. Charles, S. Wright, D. Papailiopoulos, NeurIPS 2018, [link]

Distributed Training Optimization

ICML 2018

DRACO: Robust Distributed Training via Redundant Gradients

L. Chen, H. Wang, Z. Charles, D. Papailiopoulos, ICML 2018, [link]

Distributed Training Robustness

SysML 2018

Draco: Robust Distributed Training against Adversaries

L. Chen, H. Wang, D. Papailiopoulos, SysML 2018, [link]

Distributed Training Robustness

2017

IROS 2017

Recognizing Actions during Tactile Manipulations through Force Sensing

G. Subramani, D. Rakita, H. Wang, J. Black, M. Zinn, M. Gleicher, IROS 2017, [link]

Robotics Sensing

Hongyi Wang

Publications

2025

K2-V2: A 360-Open, Reasoning-Enhanced LLM

2024

SHED: Shapley-Based Automated Dataset Refinement for Instruction Fine-Tuning

Model-GLUE: Democratized LLM Scaling for A Large Model Zoo in the Wild

FLoRA: Federated Fine-Tuning Large Language Models with Heterogeneous Low-Rank Adaptations

TrustLLM: Trustworthiness in Large Language Models

Maestro: Uncovering Low-Rank Structures via Trainable Decomposition

LLM360: Towards Fully Transparent Open-Source LLMs

Crystal: Illuminating LLM Abilities on Language and Code

RedCoast: A Lightweight Tool to Automate Distributed Training of LLMs on Any GPU/TPUs

Does compressing activations help model parallel training?

Fusing Models with Complementary Expertise

2023

FedNAR: Federated Optimization with Normalized Annealing Regularization

Cuttlefish: Low-rank Model Training without All The Tuning

MPCFormer: fast, performant and private Transformer inference with MPC

Federated Learning as Variational Inference: A Scalable Expectation Propagation Approach

2022

Efficient Federated Learning on Knowledge Graphs via Privacy-preserving Relation Embedding Aggregation

Rare Gems: Finding Lottery Tickets at Initialization

AMP: Automatically Finding Model Parallel Strategies with Heterogeneity Awareness

On the Utility of Gradient Compression in Distributed Training Systems

2021

Pufferfish: Communication-efficient Models At No Extra Cost

Accordion: Adaptive Gradient Communication via Critical Learning Regime Identification

2020

FedML: A Research Library and Benchmark for Federated Machine Learning

Attack of the Tails: Yes, You Really Can Backdoor Federated Learning

Federated Learning with Matched Averaging

2019

DETOX: A Redundancy-based Framework for Faster and More Robust Gradient Aggregation

Demonstration of Nimbus: Model-based Pricing for Machine Learning in a Data Marketplace

ErasureHead: Distributed Gradient Descent without Delays Using Approximate Gradient Coding

2018

The Effect of Network Width on the Performance of Large-batch Training

ATOMO: Communication-efficient Learning via Atomic Sparsification

DRACO: Robust Distributed Training via Redundant Gradients

Draco: Robust Distributed Training against Adversaries

2017

Recognizing Actions during Tactile Manipulations through Force Sensing