😉 About Me
I am an undergraduate student pursuing dual degrees in Information Engineering and Public Administration at Shandong University. My research interests lie in the intersection of Large Language Models (LLMs) and Software Engineering, particularly in code translation, automated program repair, and automated program understanding.
I have led multiple research projects in collaboration with Shandong University’s Research Center for Architecture and Embedded Systems and Shandong University’s Smart National Governance Laboratory, under the guidance of Dr. Zhen Yang and Prof. Liguo Fei. Additionally, my work has been fortunate to receive guidance from Prof. Zhi Jin and Prof. Ge Li at Key Laboratory of High Confidence Software Technologies, Ministry of Education (Peking University). I will soon begin my Master’s studies at Peking University, advised by Prof. Dan Hao.
I have published in top-tier venues including IEEE Transactions on Software Engineering (CCF-A), and the ACM SIGSOFT International Symposium on Software Testing and Analysis (ISSTA 2025) (CCF-A).
🔥 I am actively looking for a Internship position in AI/LLM/AI4SE. 🔥
📖 Education
Shandong University (ARWU Top 101-150)
B.Eng in Information Engineering & B.A in Public Administration
Sept 2021 - Present
🔥 News
- 2025.03: 🎉🎉 One paper accepted to the ACM SIGSOFT International Symposium on Software Testing and Analysis (ISSTA’25)!
- 2024.10: 🎉🎉 One paper accepted at IEEE Transactions on Software Engineering (TSE 2024)!
📝 Publications
* Equal contribution
Automated commit message generation with large language models: An empirical study and beyond
Pengyu Xue*, Linhao Wu*, Zhongxing Yu, Zhi Jin, Zhen Yang, Xinyi Li, Zhenyu Yang, Yue Tan.
IEEE Transactions on Software Engineering (CCF-A Journal), 2024
[PDF] | [Code]
- Curated high-quality dataset through multi-step filtering
- Led first comprehensive evaluation of LLMs for commit message generation
- A context-learning framework based on retrieval, ERICommiter, was developed to significantly improve efficiency and performance in CMG tasks
ClassEval-T: Evaluating Large Language Models in Class-Level Code Translation
Pengyu Xue*, Linhao Wu*, Zhen Yang, Chengyi Wang, Xiang Li, Yuxiang Zhang, Jia Li, Ruikai Jin, Yifei Pei, Zhaoyan Shen, Xiran Lyu, Jacky Wai Keung.
ISSTA 2025 (CCF-A Conference)，2025 [PDF] | [Code]
- Built ClassEval-T benchmark covering Python/Java/C++ (360+ person-hours)
- Designed three novel translation strategies for real-world scenarios
- A manual analysis of more than 1200 failure cases summarizes the common types of errors that LLMs makes in class-level code translation tasks
Exploring and lifting the robustness of LLM-powered automated program repair with metamorphic testing
Pengyu Xue*, Linhao Wu*, Zhen Yang, Zhongxing Yu, Zhi Jin, Ge Li, Yan Xiao, Shuo Liu, Xinyi Li, Hongyi Lin, and Jingwen Wu.
arXiv preprint arXiv:2410.07516, 2024 [PDF]
- Created MT-LAPR testing framework with 9 metamorphic relations
- Tested LLMs and ultimately revealed their robustness deficiencies in the APR task.
- Integrated CodeT5-based model improving code readability and LLMs’ robustness by 49.32%
🏆 Honors and Awards
- 2024 ICM Honorable Mention - Interdisciplinary Contest in Modeling
- 2022-2023 New 120th Anniversary Social Scholarship, Shandong University (Top 0.5%)
- 2022-2023 First-Class Outstanding Student Award, Shandong University (Top 0.8%)
🏢 Internship Experience
-
Shandong University – Research Center for Architecture and Embedded Systems
- Position: Research Assistant
- Project: Automated Commit Message Generation with Large Language Models; Class-Level Code Translation Benchmark: ClassEval-T
-
Shandong University – Smart National Governance Laboratory
- Position: Research Assistant
- Project: AI-Driven Dynamic Decision-Making for Catastrophic Emergency Response
📊 Academic Activities
- Oral Presentation: China Information Fusion Conference 2023
- Poster Presentation: National Big Data & Social Computing Conference 2024