Scalable IT Solutions Using MLP Neural Networks and AI-based Big Data Processing

Abstract

Scalability and computational efficiency are significant challenges for modern IT systems as AI and big data applications continue to grow. This study explores the use of Multilayer Perceptrons (MLPs) as a scalable solution to enhance performance in cloud and IoT environments. The proposed hybrid MLP model, integrated with Apache Spark and TensorFlow, achieved a classification accuracy of 91.81%, surpassing K-Means (87.73%) and Decision Trees (85.77%). Advanced preprocessing techniques, including normalization, dimensionality reduction, and automated hyperparameter tuning, reduced training time by 1.5 times, thereby improving computational efficiency. The model also shows strong potential in predictive analytics tasks that require fast and accurate decision-making. Despite challenges such as interpretability and computational load, future work will focus on exploring semi-supervised learning, federated learning, and explainable AI to enhance transparency and reduce dependence on labelled data. Overall, the findings suggest that MLPs provide a solid foundation for scalable and efficient IT systems in high-dimensional, real-time data environments.