An advanced pruning method in the architecture of extreme learning machines using L1-regularization and bootstrapping.
Nenhuma Miniatura disponível
Data
2020
Título da Revista
ISSN da Revista
Título de Volume
Editor
Resumo
Extreme learning machines (ELMs) are efficient for classification, regression, and time
series prediction, as well as being a clear solution to backpropagation structures to determine values
in intermediate layers of the learning model. One of the problems that an ELM may face is due
to a large number of neurons in the hidden layer, making the expert model a specific data set.
With a large number of neurons in the hidden layer, overfitting is more likely and thus unnecessary
information can deterioriate the performance of the neural network. To solve this problem, a pruning
method is proposed, called Pruning ELM Using Bootstrapped Lasso BR-ELM, which is based on
regularization and resampling techniques, to select the most representative neurons for the model
response. This method is based on an ensembled variant of Lasso (achieved through bootstrap
replications) and aims to shrink the output weight parameters of the neurons to 0 as many and as
much as possible. According to a subset of candidate regressors having significant coefficient values
(greater than 0), it is possible to select the best neurons in the hidden layer of the ELM. Finally, pattern
classification tests and benchmark regression tests of complex real-world problems are performed by
comparing the proposed approach to other pruning models for ELMs. It can be seen that statistically
BR-ELM can outperform several related state-of-the-art methods in terms of classification accuracies
and model errors (while performing equally to Pruning-ELM P-ELM), and this with a significantly
reduced number of finally selected neurons.
Descrição
Palavras-chave
Lasso with bootstrapping, Pruning of neurons, Least angle regression
Citação
SOUZA, P. V. de C. et al. An advanced pruning method in the architecture of extreme learning machines using L1-regularization and bootstrapping. Eletronics, v. 9, n. 5, 2020. Disponível em: <https://www.mdpi.com/2079-9292/9/5/811>. Acesso em: 29 abr. 2022.