Title: Effective Automation of REST API Testing with Machine Learning and Language Models
Date: April 2nd, 2025
Time: 1:00 PM - 3:00 PM EST
Location: Virtual (https://gatech.zoom.us/my/codingsoo)
Committee:
Dr. Alessandro Orso (Advisor) - School of Computer Science, Georgia Institute of Technology
Dr. Qirun Zhang - School of Computer Science, Georgia Institute of Technology
Dr. Spencer Rugaber - College of Computing, Georgia Institute of Technology
Dr. Manish Motwani - School of Electrical Engineering and Computer Science, Oregon State University
Dr. Saurabh Sinha - IBM Research
Abstract:
REST APIs are fundamental to modern web services, necessitating rigorous testing to ensure their reliability. While the OpenAPI Specification provides a structured framework for API documentation, existing black-box testing tools struggle with limited code coverage and fault detection. To address these challenges, this dissertation advances REST API testing by incorporating machine learning and natural language processing (NLP) techniques to enhance automated test generation and execution.
Through an empirical evaluation of ten state-of-the-art black-box testing tools across twenty RESTful web services, key limitations in coverage and fault detection were identified. Based on these insights, this work introduces several innovations: (1) leveraging NLP techniques to systematically extract testable rules from human-readable API documentation, improving the generation of meaningful test cases; (2) utilizing Large Language Models (LLMs) to refine API specifications, detect nuanced constraints, and generate realistic input values for testing; (3) applying Reinforcement Learning (RL) to dynamically prioritize and optimize test execution based on API response feedback, leading to improved efficiency and effectiveness; (4) integrating fine-tuned Small Language Models (SLMs) to generate realistic input values and resolve parameter dependencies efficiently, significantly improving code coverage and fault detection while ensuring computational efficiency; and (5) developing a multi-agent black-box testing approach that coordinates multiple specialized agents to explore API behaviors more effectively, increasing test coverage and fault detection.
By integrating these advancements, this dissertation significantly improves the scalability, efficiency, and effectiveness of black-box REST API testing. The proposed machine learning-driven multi-agent framework optimizes test generation and execution, surpassing existing tools in operation coverage, code coverage, and fault detection. I believe these contributions advance the field by making automated REST API testing more adaptive and comprehensive for real-world web services.