1. STORRE
  2. Faculty of Natural Sciences
  3. Computing Science and Mathematics
  4. Computing Science and Mathematics Conference Papers and Proceedings
Please use this identifier to cite or link to this item: http://hdl.handle.net/1893/35519
Full metadata record
DC FieldValueLanguage
dc.contributor.authorSobania, Dominiken_UK
dc.contributor.authorGeiger, Alinaen_UK
dc.contributor.authorCallan, Jamesen_UK
dc.contributor.authorBrownlee, Alexanderen_UK
dc.contributor.authorHanna, Carolen_UK
dc.contributor.authorMoussa, Rebeccaen_UK
dc.contributor.authorZamorano López, Maren_UK
dc.contributor.authorPetke, Justynaen_UK
dc.contributor.authorSarro, Federicaen_UK
dc.date.accessioned2023-11-05T01:00:30Z-
dc.date.available2023-11-05T01:00:30Z-
dc.identifier.urihttp://hdl.handle.net/1893/35519-
dc.description.abstractLarge language models (LLMs) have recently been integrated in a variety of applications including software engineering tasks. In this work, we study the use of LLMs to enhance the explainability of software patches. In particular, we evaluate the performance of GPT 3.5 in explaining patches generated by the search-based automated program repair system ARJA-e for 30 bugs from the popular Defects4J benchmark. We also investigate the performance achieved when explaining the corresponding patches written by software developers. We find that on average 84% of the LLM explanations for machine-generated patches were correct and 54% were complete for the studied categories in at least 1 out of 3 runs. Furthermore, we find that the LLM generates more accurate explanations for machine-generated patches than for human-written ones.en_UK
dc.language.isoenen_UK
dc.relationSobania D, Geiger A, Callan J, Brownlee A, Hanna C, Moussa R, Zamorano López M, Petke J & Sarro F (2023) Evaluating Explanations for Software Patches Generated by Large Language Models. In: Symposium on Search-Based Software Engineering- Challenge Track, San Francisco, CA, USA, 08.12.2023-08.12.2023.en_UK
dc.rightsThis item has been embargoed for a period. During the embargo please use the Request a Copy feature at the foot of the Repository record to request a copy directly from the author. You can only request a copy if you wish to use this work for your own research or private study.en_UK
dc.rights.urihttp://creativecommons.org/licenses/by/4.0/en_UK
dc.subjectLarge Language Modelsen_UK
dc.subjectSoftware Patchesen_UK
dc.subjectAI Explainabilityen_UK
dc.subjectProgram Repairen_UK
dc.subjectGenetic Improvementen_UK
dc.titleEvaluating Explanations for Software Patches Generated by Large Language Modelsen_UK
dc.typeConference Paperen_UK
dc.rights.embargodate2025-09-30en_UK
dc.rights.embargoreason[ssbse23challenge-final19.pdf] Until this work is published there will be an embargo on the full text of this worken_UK
dc.type.statusAM - Accepted Manuscripten_UK
dc.contributor.funderEngineering and Physical Sciences Research Councilen_UK
dc.contributor.funderEuropean Commission (Horizon 2020)en_UK
dc.author.emailalexander.brownlee@stir.ac.uken_UK
dc.citation.conferencedates2023-12-08 - 2023-12-08en_UK
dc.citation.conferencelocationSan Francisco, CA, USAen_UK
dc.citation.conferencenameSymposium on Search-Based Software Engineering- Challenge Tracken_UK
dc.contributor.affiliationJohannes Gutenberg University of Mainzen_UK
dc.contributor.affiliationJohannes Gutenberg University of Mainzen_UK
dc.contributor.affiliationUniversity College Londonen_UK
dc.contributor.affiliationComputing Science and Mathematics - Divisionen_UK
dc.contributor.affiliationUniversity College Londonen_UK
dc.contributor.affiliationUniversity College Londonen_UK
dc.contributor.affiliationUniversity College Londonen_UK
dc.contributor.affiliationUniversity College Londonen_UK
dc.contributor.affiliationUniversity College Londonen_UK
dc.identifier.wtid1946172en_UK
dc.contributor.orcid0000-0003-2892-5059en_UK
dc.date.accepted2023-09-30en_UK
dcterms.dateAccepted2023-09-30en_UK
dc.date.filedepositdate2023-10-17en_UK
rioxxterms.apcnot requireden_UK
rioxxterms.typeConference Paper/Proceeding/Abstracten_UK
rioxxterms.versionAMen_UK
local.rioxx.authorSobania, Dominik|en_UK
local.rioxx.authorGeiger, Alina|en_UK
local.rioxx.authorCallan, James|en_UK
local.rioxx.authorBrownlee, Alexander|0000-0003-2892-5059en_UK
local.rioxx.authorHanna, Carol|en_UK
local.rioxx.authorMoussa, Rebecca|en_UK
local.rioxx.authorZamorano López, Mar|en_UK
local.rioxx.authorPetke, Justyna|en_UK
local.rioxx.authorSarro, Federica|en_UK
local.rioxx.projectProject ID unknown|Engineering and Physical Sciences Research Council|http://dx.doi.org/10.13039/501100000266en_UK
local.rioxx.projectProject ID unknown|European Commission (Horizon 2020)|en_UK
local.rioxx.freetoreaddate2025-09-30en_UK
local.rioxx.licencehttp://www.rioxx.net/licenses/under-embargo-all-rights-reserved||2025-09-30en_UK
local.rioxx.licencehttp://creativecommons.org/licenses/by/4.0/|2025-09-30|en_UK
local.rioxx.filenamessbse23challenge-final19.pdfen_UK
local.rioxx.filecount1en_UK
Appears in Collections:Computing Science and Mathematics Conference Papers and Proceedings

Files in This Item:
File Description SizeFormat 
ssbse23challenge-final19.pdfFulltext - Accepted Version295.27 kBAdobe PDFUnder Embargo until 2025-09-30    Request a copy


This item is protected by original copyright

View License
Show simple item record


A file in this item is licensed under a Creative Commons License Creative Commons

Items in the Repository are protected by copyright, with all rights reserved, unless otherwise indicated.

The metadata of the records in the Repository are available under the CC0 public domain dedication: No Rights Reserved https://creativecommons.org/publicdomain/zero/1.0/

If you believe that any material held in STORRE infringes copyright, please contact library@stir.ac.uk providing details and we will remove the Work from public display in STORRE and investigate your claim.