The kiosk still has to be fair

A Comparative Case Study of Supervised and Unsupervised Machine Learning Approaches in Artificial Intelligence: Applications to Multimodal Customer Service Systems

Abstract

This case study compares two supervised learning approaches — Support Vector Machines (SVMs) and Convolutional Neural Networks (CNNs) — with two unsupervised approaches — K-Means clustering and Principal Component Analysis (PCA) — in the context of multimodal artificial intelligence systems for customer service. The selected methods are widely applied in face detection and speech recognition pipelines that power intelligent kiosks, virtual agents, and contact-centre analytics. The report outlines the background and significance of these techniques, states clear aims and objectives, describes the models and algorithms in detail, and provides a structured comparison across complexity, efficiency and accuracy. Economic and social challenges, including labour displacement in contact centres, biometric bias, privacy risks and accessibility benefits, are examined critically. Findings indicate that supervised methods generally deliver higher accuracy when labelled data are available, while unsupervised methods offer valuable exploratory power and lower labelling costs. The analysis prepares the ground for practical multimodal system development and highlights the need for careful governance when deploying biometric AI in customer-facing environments.

Introduction

Artificial intelligence has moved rapidly from research laboratories into everyday customer service environments. Intelligent kiosks that recognise faces for personalised greetings, voice-enabled virtual agents that transcribe and interpret spoken queries, and analytics platforms that cluster customer interactions for insight are no longer experimental; they are operational in banks, telecommunications firms, airlines and government services centres. These systems rely on a combination of computer vision and natural language processing, two domains in which both supervised and unsupervised machine learning play essential roles.

Supervised learning algorithms learn from labelled examples to map inputs to known outputs, making them well suited to classification tasks such as identifying a face or recognising a spoken command. Unsupervised algorithms discover structure in unlabelled data, enabling dimensionality reduction, clustering of similar customer behaviours, and exploratory analysis of large interaction logs. Understanding the relative strengths, limitations and trade-offs of these families of methods is therefore critical for practitioners designing multimodal customer-service systems.

This case study examines four representative approaches — Support Vector Machines and Convolutional Neural Networks on the supervised side, and K-Means clustering together with Principal Component Analysis on the unsupervised side. The discussion is framed around their application to face detection and speech recognition components that support modern customer-service interfaces. The analysis also addresses the economic and social challenges that accompany the deployment of such technologies.

Significance of the case study

The significance of this comparative analysis is both practical and academic. From a practical perspective, customer-service organisations face rising volumes of digital and voice interactions while simultaneously seeking to reduce operational cost and improve customer experience. Multimodal AI systems that combine visual and auditory channels can deliver faster, more personalised service, yet the choice of underlying algorithms materially affects accuracy, computational cost, data requirements and risk profile.

Academically, the case study addresses the subject learning outcomes of differentiating AI algorithms and methods, explaining their real-world application, and describing components relevant to natural language processing and vision systems. By selecting techniques commonly employed in face detection and speech recognition, the work also provides a conceptual foundation for subsequent practical development of a multimodal interaction system.

Finally, the study contributes to critical awareness of the broader societal implications of biometric and conversational AI. Customer-service deployments of face and speech technologies raise questions of fairness, privacy, employment impact and accessibility that cannot be ignored if organisations are to adopt AI responsibly.

Aims and objectives

The overall aim of this case study is to compare selected supervised and unsupervised machine learning approaches and to evaluate their suitability for multimodal AI applications in customer service, with particular reference to face detection and speech recognition.

1. Describe two supervised learning approaches (SVMs and CNNs), including their underlying models and algorithms, and illustrate their application to face detection and speech-related tasks in customer service.
2. Describe two unsupervised learning approaches (K-Means clustering and PCA), including their models and algorithms, and illustrate their use in exploratory analysis and pre-processing for the same domains.
3. Compare the four approaches systematically in terms of complexity, efficiency and accuracy.
4. Analyse key economic and social challenges associated with the deployment of these AI methods in customer-service contexts.
5. Draw conclusions that inform both theoretical understanding and practical system design.

Supervised learning

Support Vector Machines. SVMs are supervised classifiers that seek a hyperplane maximising the margin between classes in a high-dimensional feature space (Cortes & Vapnik, 1995). The decision boundary is determined by a subset of training examples known as support vectors. Non-linear problems are handled via the kernel trick, which implicitly maps data into a higher-dimensional space without explicit computation of the mapping.

In customer-service applications, SVMs have been applied both to face detection (as binary classifiers distinguishing face from non-face patches) and to speech-related tasks such as speaker verification or intent classification from acoustic features. Their strengths include strong theoretical foundations, effectiveness in high-dimensional spaces, and relatively modest data requirements compared with deep networks. Limitations include sensitivity to the choice of kernel and regularisation parameters, and reduced scalability on very large datasets.

Convolutional Neural Networks. CNNs are deep learning architectures that apply learnable filters across spatial or temporal dimensions, followed by non-linear activation and pooling operations (LeCun et al., 2015). Hierarchical feature learning allows early layers to detect simple patterns (edges, phoneme-like units) while deeper layers capture complex structures such as facial landmarks or higher-level speech representations.

CNNs dominate modern face detection pipelines; detectors such as those based on single-shot multi-box detection or RetinaFace rely on convolutional backbones to achieve high accuracy across scales and poses (Deng et al., 2020). In speech processing, CNNs operating on spectrograms or raw waveforms contribute to acoustic modelling and keyword spotting. Their principal advantages are superior accuracy on large labelled datasets and the ability to learn features end-to-end. Drawbacks include high computational and data demands during training, and reduced interpretability relative to linear models.

Unsupervised learning

K-Means clustering. K-Means is a centroid-based clustering algorithm that partitions a dataset into k groups by iteratively assigning points to the nearest centroid and updating centroids as the mean of assigned points (Lloyd, 1982). The objective is to minimise within-cluster sum of squared distances.

In multimodal customer-service systems, K-Means is valuable for exploratory analysis: clustering acoustic embeddings to discover common query types, grouping facial embeddings for identity or demographic analysis (subject to ethical constraints), or segmenting customer interaction patterns for personalisation. It is computationally efficient and easy to implement, yet sensitive to the choice of k and initialisation, and assumes roughly spherical clusters of similar size.

Principal Component Analysis. PCA is a linear dimensionality-reduction technique that projects data onto orthogonal axes of maximum variance (Pearson, 1901; Hotelling, 1933). By retaining only the leading principal components, PCA compresses data while preserving most of the variance, and can also serve as a denoising step.

Historically, PCA underpinned the Eigenfaces method for face recognition (Turk & Pentland, 1991). In contemporary pipelines it remains useful as a pre-processing stage that reduces feature dimensionality before supervised classification or clustering, thereby improving computational efficiency and mitigating the curse of dimensionality. In speech processing, PCA or related techniques are applied to acoustic feature vectors. PCA is computationally efficient and deterministic, but captures only linear relationships and may discard subtle discriminative information if too few components are retained.

Comparison

Supervised methods (particularly CNNs) currently deliver the highest accuracy for core recognition tasks when labelled data are plentiful, at the cost of greater complexity and training expense. Unsupervised methods excel at data exploration, dimensionality reduction and reducing the need for expensive annotation. Hybrid pipelines that use PCA or clustering as pre-processing stages before supervised classifiers are common in production systems and often represent the most pragmatic balance.

Economic and social challenges

The deployment of face detection and speech recognition technologies in customer service generates both economic opportunity and significant social risk. On the economic side, automation of routine enquiries can reduce average handling time and operating cost, while personalisation enabled by face or voice recognition can improve conversion and satisfaction metrics. However, these gains are unevenly distributed. Contact-centre roles that consist largely of scripted interactions are among the most exposed to automation; displacement effects are already observable in some markets and raise questions of transitional support, reskilling and the quality of the remaining human roles (International Labour Organization, 2024).

Social challenges centre on bias, privacy and accessibility. Face recognition systems have historically exhibited higher error rates for certain demographic groups, a problem that can translate into discriminatory service experiences if not rigorously audited (Buolamwini & Gebru, 2018). Speech recognition accuracy likewise varies with accent, dialect and background noise, potentially disadvantaging non-native speakers or customers in noisy environments. Privacy concerns are acute: continuous or opportunistic capture of facial images and voice recordings creates sensitive biometric datasets whose secondary use, retention and security must be governed carefully under privacy legislation and emerging AI regulation.

At the same time, well-designed multimodal systems can advance inclusion. Speech interfaces assist customers with visual impairments or literacy barriers; face-based authentication can reduce friction for users who struggle with passwords. Realising these benefits while mitigating harms requires deliberate design choices, diverse training data, ongoing fairness evaluation, transparent consent mechanisms and human oversight for high-stakes decisions. Organisations that treat these considerations as peripheral rather than integral risk both reputational damage and regulatory sanction.

Longer-term risks include over-reliance on proprietary models, concentration of capability among a small number of technology providers, and the environmental cost of training and serving large neural networks. These systemic issues reinforce the need for critical evaluation rather than uncritical adoption.

Conclusion

This case study has compared Support Vector Machines and Convolutional Neural Networks as representative supervised approaches with K-Means clustering and Principal Component Analysis as representative unsupervised approaches. The analysis demonstrates that supervised methods currently offer superior accuracy for face detection and speech recognition tasks when adequate labelled data are available, while unsupervised methods provide essential tools for exploration, dimensionality reduction and cost-effective pre-processing. Complexity and efficiency trade-offs further favour hybrid architectures in many practical customer-service settings.

Economic benefits in the form of productivity and personalisation must be weighed against labour-market disruptions, biometric bias, privacy exposure and accessibility implications. Responsible deployment therefore requires not only technical competence in algorithm selection but also organisational commitment to fairness, transparency and continuous evaluation.

The comparative understanding developed here supplies a direct conceptual foundation for the design and implementation of a multimodal system that combines computer vision and speech recognition — work that constitutes the next stage of practical application in this subject.

References

• Buolamwini, J., & Gebru, T. (2018). Gender shades: Intersectional accuracy disparities in commercial gender classification. Proceedings of Machine Learning Research, 81, 1–15.
• Cortes, C., & Vapnik, V. (1995). Support-vector networks. Machine Learning, 20(3), 273–297.
• Deng, J., Guo, J., Ververas, E., Kotsia, I., & Zafeiriou, S. (2020). RetinaFace: Single-shot multi-level face localisation in the wild. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 5203–5212.
• Hotelling, H. (1933). Analysis of a complex of statistical variables into principal components. Journal of Educational Psychology, 24(6), 417–441.
• International Labour Organization. (2024). Mind the AI divide: Shaping a global perspective on the future of work. ILO / United Nations.
• LeCun, Y., Bengio, Y., & Hinton, G. (2015). Deep learning. Nature, 521(7553), 436–444.
• Lloyd, S. (1982). Least squares quantization in PCM. IEEE Transactions on Information Theory, 28(2), 129–137.
• Pearson, K. (1901). On lines and planes of closest fit to systems of points in space. Philosophical Magazine, 2(11), 559–572.
• Turk, M., & Pentland, A. (1991). Eigenfaces for recognition. Journal of Cognitive Neuroscience, 3(1), 71–86.