An Algorithm for Exploratory Analysis and Normalization of Big Data with Pandas
Keywords:big data, data mining, big data analysis, educational analytics, natural language processing
The digitization of business processes and the extraction of answers to user requests for big data are modern problems that are of great interest to scientists and researchers. The data generated so far, located in various corpora, is much more than can be analyzed. Therefore, they are collected, identified, cleaned and normalized to be used most adequately. Segmentation, assumptions and hypotheses contribute to the degree of satisfaction with the returned result. The research proposed a general method for collecting, cleaning and normalizing data from various sources, structurally modelling it into appropriate models, then testing hypotheses and analyzing the obtained results to conclude large academic data that will benefit the business in making management decisions. This is possible with the means of computational linguistics and with the help of Python data manipulation libraries.
How to Cite
LicenseCopyright (c) 2023 Proceedings of the Bulgarian Academy of Sciences
Copyright (c) 2022 Proceedings of the Bulgarian Academy of Sciences
Copyright is subject to the protection of the Bulgarian Copyright and Associated Rights Act. The copyright holder of all articles on this site is Proceedings of the Bulgarian Academy of Sciences. If you want to reuse any part of the content, please, contact us.