Analysis of the effectiveness of machine learning algorithms in big data processing

Abstract

The aim of this article is to analyze the effectiveness of machine learning algorithms and identify optimal approaches for their use under heavy loads and large volumes of information. Special attention is paid to tasks requiring high accuracy and speed, such as financial forecasting, medical diagnostics, and behavioral data analysis. The research methodology included a comparative analysis of various machine learning algorithms, such as linear regression, decision trees, support vector machines, and deep neural networks. Key factors influencing the speed and accuracy of data processing were evaluated, including data size, model complexity, computational resources, and data quality. A series of experiments on real-world datasets was conducted to assess each algorithm’s performance in terms of accuracy, training time, and computational resource demands. The results showed that deep neural networks provide high accuracy on unstructured data, but require significant computational resources and training time. Algorithms such as linear regression and decision trees demonstrated high processing speed on simpler datasets, though their accuracy decreases as task complexity increases. The support vector machine method proved effective for classification and prediction tasks, particularly in financial and medical applications. Random forests were found to be efficient for text classification, providing a balance between speed and accuracy. The conclusions indicate that the choice of algorithm depends on the task’s specific characteristics, data size, and accuracy requirements. Optimizing models for distributed computing environments is a key direction for improving productivity, as it allows for parallelization and reduces training time