Home/News

Study reveals new methods to improve ‘plug-and-play’ ML components

News / A2I2 Media / June 28, 2021

Dr Alex Cummaudo of the Applied Artificial Intelligence Institute (A²I²) aims to improve the quality of software that interacts with pre-trained machine learning (ML) models offered by cloud vendors. Over the past two years his research has developed an innovative framework that can be adopted to make usage of such AI components more reliable.

Multiple leading cloud vendors offer pre-trained ML models to software developers via their web service APIs. Common examples of this technology include object detection in images or speech-to-text services. These ‘intelligent’ web services, often marketed as ‘plug-and-play’ AI, aim to reduce technical knowledge required to infuse AI into applications, thereby reducing time-to-market. 

A deeper look into how the services work suggests that the technology isn’t as simple as it looks. Dr Cummaudo’s research has highlighted that there are undocumented risks in how these services behave over time, with many developers facing complicated issues they cannot easily resolve.

“Our study has shown that AI-empowered cloud services aren’t as robust as people think. What is needed is more clear communication from the cloud vendors that empowers software engineers to build robust software by discussing potential risks,” Dr Cummaudo said.

“Inconsistencies can result in reliability issues, reducing software quality, and the lack of proper understanding by developers can lead to lost productivity in software teams.”

To overcome these issues, a facade architecture was developed and tested over a five-month study against a prominent computer vision service. Designed to assist developers in handling service behaviour, the approach identified 331 cases of evolution that would have otherwise ‘leaked’ into client systems and affected end-users. Dr Cummaudo is confident that this research will be used to better inform developers on such technology risks. 

“This research aims to better inform industry professionals on how they can use intelligent web services to maintain overall system reliability in their development of complex applications.” 

The findings from the study have been published in a series of top-tier software engineering journals and conferences. For more information, read Dr Cummaudo’s blog post and the peer-reviewed articles below.