Please use this identifier to cite or link to this item: http://hdl.handle.net/1893/24942
Full metadata record
DC FieldValueLanguage
dc.contributor.authorScardapane, Simoneen_UK
dc.contributor.authorComminiello, Daniloen_UK
dc.contributor.authorHussain, Amiren_UK
dc.contributor.authorUncini, Aurelioen_UK
dc.date.accessioned2017-05-16T00:31:51Z-
dc.date.available2017-05-16T00:31:51Zen_UK
dc.date.issued2017-06-07en_UK
dc.identifier.urihttp://hdl.handle.net/1893/24942-
dc.description.abstractIn this paper, we address the challenging task of simultaneously optimizing (i) the weights of a neural network, (ii) the number of neurons for each hidden layer, and (iii) the subset of active input features (i.e., feature selection). While these problems are traditionally dealt with separately, we propose an efficient regularized formulation enabling their simultaneous parallel execution, using standard optimization routines. Specifically, we extend the group Lasso penalty, originally proposed in the linear regression literature, to impose group-level sparsity on the network's connections, where each group is defined as the set of outgoing weights from a unit. Depending on the specific case, the weights can be related to an input variable, to a hidden neuron, or to a bias unit, thus performing simultaneously all the aforementioned tasks in order to obtain a compact network. We carry out an extensive experimental evaluation, in comparison with classical weight decay and Lasso penalties, both on a toy dataset for handwritten digit recognition, and multiple realistic mid-scale classification benchmarks. Comparative results demonstrate the potential of our proposed sparse group Lasso penalty in producing extremely compact networks, with a significantly lower number of input features, with a classification accuracy which is equal or only slightly inferior to standard regularization terms.en_UK
dc.language.isoenen_UK
dc.publisherElsevieren_UK
dc.relationScardapane S, Comminiello D, Hussain A & Uncini A (2017) Group Sparse Regularization for Deep Neural Networks. Neurocomputing, 241, pp. 81-89. https://doi.org/10.1016/j.neucom.2017.02.029en_UK
dc.rightsThis item has been embargoed for a period. During the embargo please use the Request a Copy feature at the foot of the Repository record to request a copy directly from the author. You can only request a copy if you wish to use this work for your own research or private study. Accepted refereed manuscript of: Scardapane S, Comminiello D, Hussain A & Uncini A (2017) Group Sparse Regularization for Deep Neural Networks, Neurocomputing, 241, pp. 81-89. DOI: 10.1016/j.neucom.2017.02.029 © 2017, Elsevier. Licensed under the Creative Commons Attribution-NonCommercial-NoDerivatives 4.0 International http://creativecommons.org/licenses/by-nc-nd/4.0/en_UK
dc.rights.urihttp://creativecommons.org/licenses/by-nc-nd/4.0/en_UK
dc.subjectDeep networksen_UK
dc.subjectGroup sparsityen_UK
dc.subjectPruningen_UK
dc.subjectFeature selectionen_UK
dc.titleGroup Sparse Regularization for Deep Neural Networksen_UK
dc.typeJournal Articleen_UK
dc.rights.embargodate2018-02-11en_UK
dc.rights.embargoreason[Scardapane_etal_Manuscript.pdf] Publisher requires embargo of 12 months after formal publication.en_UK
dc.identifier.doi10.1016/j.neucom.2017.02.029en_UK
dc.citation.jtitleNeurocomputingen_UK
dc.citation.issn0925-2312en_UK
dc.citation.volume241en_UK
dc.citation.spage81en_UK
dc.citation.epage89en_UK
dc.citation.publicationstatusPublisheden_UK
dc.citation.peerreviewedRefereeden_UK
dc.type.statusAM - Accepted Manuscripten_UK
dc.author.emailahu@cs.stir.ac.uken_UK
dc.citation.date10/02/2017en_UK
dc.contributor.affiliationSapienza University of Romeen_UK
dc.contributor.affiliationSapienza University of Romeen_UK
dc.contributor.affiliationComputing Scienceen_UK
dc.contributor.affiliationSapienza University of Romeen_UK
dc.identifier.isiWOS:000398752700008en_UK
dc.identifier.scopusid2-s2.0-85013055161en_UK
dc.identifier.wtid536294en_UK
dc.contributor.orcid0000-0002-8080-082Xen_UK
dc.date.accepted2017-02-07en_UK
dc.date.filedepositdate2017-02-08en_UK
Appears in Collections:Computing Science and Mathematics Journal Articles

Files in This Item:
File Description SizeFormat 
Scardapane_etal_Manuscript.pdfFulltext - Accepted Version564.69 kBAdobe PDFView/Open


This item is protected by original copyright



A file in this item is licensed under a Creative Commons License Creative Commons

Items in the Repository are protected by copyright, with all rights reserved, unless otherwise indicated.

If you believe that any material held in STORRE infringes copyright, please contact library@stir.ac.uk providing details and we will remove the Work from public display in STORRE and investigate your claim.