Xupeng Miao

Xupeng Miao is currently a Post Doctoral Fellow working with Prof. Zhihao Jia and Prof. Tianqi Chen in Catalyst Group and Parallel Data Lab at Computer Science Department of Carnegie Mellon University. He is broadly interested in machine learning systems, data management and distributed computing. He is the creator of Hetu, a highly efficient distributed deep learning system, and continuously leading the team development, welcome to join us!

Before that, he received his Ph.D. degree in computer science from Peking University in June 2022, supervised by Prof. Bin Cui. He was a research intern in System Research Group of Microsoft Research Asia (MSRA), supervised by Dr. Jilong Xue. Besides, he has accumulated for more than 5 years industrial internship experience at the Machine Learning & Data Platform Department of Tencent.

news

Jan 30, 2023	I was grateful to be awared 2022 ACM China Doctoral Dissertation Award.
Oct 16, 2022	One paper was accepted by VLDB 2023.
Oct 10, 2022	One paper was accepted by VLDBJ.
Sep 29, 2022	I was awared 2022 ACM SIGMOD China Doctoral Dissertation Award!
Sep 3, 2022	I was awared WAIC 2022 Yunfan Award · Rising Stars!

selected publications

VLDB

Galvatron: Efficient Transformer Training over Multiple GPUs Using Automatic Parallelism

Xupeng Miao, Yujie Wang, Youhe Jiang, Chunan Shi and 3 more authors

Proc. VLDB Endow. 2023

Bib PDF CCF-A

@article{miao2023galvatron,
  title = {Galvatron: Efficient Transformer Training over Multiple GPUs Using Automatic Parallelism},
  author = {Miao, Xupeng and Wang, Yujie and Jiang, Youhe and Shi, Chunan and Nie, Xiaonan and Zhang, Hailin and Cui, Bin},
  journal = {Proc. {VLDB} Endow.},
  volume = {16},
  number = {3},
  pages = {470--479},
  year = {2023},
  doi = {10.14778/3570690.3570697},
  publisher = {VLDB Endowment},
}

SCIS

Hetu: A highly efficient automatic parallel distributed deep learning system

Xupeng Miao, Xiaonan Nie, Hailin Zhang, Tong Zhao and 1 more author

Sci. China Inf. Sci. 2022

Bib PDF CCF-T1

@article{DBLP:journals/chinaf/MiaoXP22,
  author = {Miao, Xupeng and Nie, Xiaonan and Zhang, Hailin and Zhao, Tong and Cui, Bin},
  title = {Hetu:  A highly efficient automatic parallel distributed deep learning system},
  journal = {Sci. China Inf. Sci.},
  url = {http://engine.scichina.com/doi/10.1007/s11432-022-3581-9},
  doi = {10.1007/s11432-022-3581-9},
  year = {2022},
}

VLDB

HET: Scaling out Huge Embedding Model Training via Cache-enabled Distributed Framework (Best Scalable Data Science Paper)

Xupeng Miao, Hailin Zhang, Yining Shi, Xiaonan Nie and 3 more authors

Proc. VLDB Endow. 2022

Bib PDF CCF-A

@article{miao2021het,
  title = {{HET:} Scaling out Huge Embedding Model Training via Cache-enabled Distributed Framework},
  author = {Miao, Xupeng and Zhang, Hailin and Shi, Yining and Nie, Xiaonan and Yang, Zhi and Tao, Yangyu and Cui, Bin},
  journal = {Proc. {VLDB} Endow.},
  volume = {15},
  number = {2},
  pages = {312--320},
  year = {2022},
  publisher = {VLDB Endowment},
  notes = {(Best Scalable Data Science Paper)},
}

SIGMOD

HET-GMP: A Graph-based System Approach to Scaling Large Embedding Model Training

Xupeng Miao, Yining Shi, Hailin Zhang, Xin Zhang and 3 more authors

In Proceedings of SIGMOD Conference 2022

Bib PDF CCF-A

@inproceedings{miao2022hetgmp,
  author = {Miao, Xupeng and Shi, Yining and Zhang, Hailin and Zhang, Xin and Nie, Xiaonan and Yang, Zhi and Cui, Bin},
  title = {{HET-GMP:} {A} Graph-based System Approach to Scaling Large Embedding Model Training},
  booktitle = {Proceedings of SIGMOD Conference},
  pages = {470--480},
  publisher = {{ACM}},
  year = {2022},
  doi = {10.1145/3514221.3517902},
}

SIGMOD

Heterogeneity-Aware Distributed Machine Learning Training via Partial Reduce

Xupeng Miao, Xiaonan Nie, Yingxia Shao, Zhi Yang and 3 more authors

In Proceedings of SIGMOD Conference 2021

Bib PDF CCF-A

@inproceedings{DBLP:conf/sigmod/MiaoNSYJM021,
  author = {Miao, Xupeng and Nie, Xiaonan and Shao, Yingxia and Yang, Zhi and Jiang, Jiawei and Ma, Lingxiao and Cui, Bin},
  title = {Heterogeneity-Aware Distributed Machine Learning Training via Partial
                 Reduce},
  booktitle = {Proceedings of SIGMOD Conference},
  pages = {2262--2270},
  publisher = {{ACM}},
  year = {2021},
  doi = {10.1145/3448016.3452773},
}