A review of unsupervised k-value selection techniques in clustering algorithms

Abstract

Purpose: Automatic grouping of data according to certain characteristics is made possible by clustering algorithms, which makes them an essential tool when working with large datasets. However, although they are unsupervised tools, they generally require the specification of the number of clusters to be formed, k, a task that may be simple for a human, but quite complex to automate. Despite the most commonly used k-value selection techniques offer acceptable results, they are not without shortcomings, suggesting that there is ample room for improvement. This paper briefly introduces clustering techniques, discusses the main shortcomings of conventional k-value selection techniques and examines the advantages and limitations of nine promising alternatives presented in recent years.

Design/methodology/approach: An evaluation of the main shortcomings of classic k-value estimation techniques has been carried out, and the newest proposals have been explained and compared.

Findings: New k-value estimation indices and methodologies proposed by authors guarantee better results, extending the use of these techniques to large volumes of data, and complex shapes and structures. However, no generical methodology able to overcome all the described shortcomings has still been developed.

Research limitations/implications: This research is limited to the newest developed techniques for k-value estimation, including proposals published since 2019. Older proposals have not been considered, as the newest ones overcome the former’s shortcomings. A k-value estimation techniques review published in 2019 is cited in the test as a base reference.

Practical implications: Although the examples listed in the text apply to industry, the techniques described and discussed in this review are applicable to any area of science that can benefit from the use of clustering techniques.

Originality/value: To date, there has been no paper comparing the new k-value estimation techniques. Although there are literature reviews comparing the classical methods, these methods are nowadays nearly obsolete due to the complexity of the data usually faced.