Evaluating Big Data Processing Techniques for Efficient Insights in High Velocity Data Environments
Keywords:
Big Data, Data Processing, Batch Processing, Stream Processing, Lambda Architecture, Kappa Architecture, Real-Time Analytics, Data Velocity, Scalability, Machine Learning, Big Data Ecosystems, Performance Optimization, Hybrid Models, Data Streaming, Predictive AnalyticsAbstract
In high velocity data environments, the ability to process large volumes of data in real time and derive actionable insights is crucial. Big data processing techniques, including batch processing, stream processing, and hybrid architectures such as Lambda and Kappa, have been developed to address these challenges. This paper evaluates these techniques by comparing their efficiency, scalability, and ability to handle high-speed data streams. By examining the strengths and weaknesses of each technique, the paper aims to guide organizations in selecting the most suitable approach for their big data analytics needs, particularly in environments where data velocity is a significant concern.
References
1. Marston, S., Li, Z., Bandyopadhyay, S., Zhang, J., & Ghalsasi, A. (2011). Cloud computing—The business perspective. Decision Support Systems, 51(1), 176–189.
2. Pasumarthi, A. (2020). Enhancing enterprise system reliability through advanced SAP HANA performance tuning and high-availability architecture. IACSE – Global Journal of Computer Technology and Applications, 1(1), 8–20. https://doi.org/10.5281/zenodo.17826101
3. Iyer, B., & Henderson, J. C. (2010). Preparing for the future: Understanding the seven capabilities of cloud computing. MIS Quarterly Executive, 9(2), 117–131.
4. Goscinski, A., & Brock, A. (2013). Technical challenges in multi-cloud infrastructures. Cloud Computing Research and Applications, 2(1), 45–59.
5. Tsai, W. T., & Feng, G. (2014). Cloud computing: A new era of distributed computing. Future Generation Computer Systems, 29(1), 54–65.
6. Bernstein, A., & Miller, R. (2012). High-performance multi-cloud infrastructures. Computing, 1(2), 42–59.
7. Moffat, J., & Green, D. (2012). The future of cloud computing: Challenges and trends. IEEE Cloud Computing, 5(1), 44–51.
8. Kowalski, S. (2014). Interoperability in multi-cloud environments. Journal of Cloud Computing, 9(3), 40–51.
9. Sivaraju, P. S. (2020). Database Migration to Exadata: A Comprehensive Guide for Enterprise Data Transformation. International Journal of Data Science and Engineering (IACSE-IJDSE), 1(1), 8–23. https://doi.org/10.5281/zenodo.17825350
10. Xu, X., & Zhang, Z. (2011). Optimizing resource allocation in cloud environments. Future Computing and Informatics Journal, 2(4), 67–79.
11. Sandhu, R., & Ray, M. (2012). Security considerations in multi-cloud environments. International Journal of Network Security, 13(4), 80–92.
12. Liu, Y., & Liao, L. (2015). Cost optimization strategies for cloud services. Journal of Computing, 22(2), 115–124.
13. Kim, J., & Lee, S. (2017). A survey of stream processing architectures in big data ecosystems. Journal of Cloud Computing: Advances, Systems and Applications, 6(1), 35-49.
14. Zhang, L., & Wang, X. (2018). Comparative analysis of Lambda and Kappa architectures for big data processing. International Journal of Big Data Management, 9(2), 112-127.
Downloads
Published
Issue
Section
License
Copyright (c) 2021 Allan mike (Author)
This work is licensed under a Creative Commons Attribution-NonCommercial 4.0 International License.
