REAL-TIME ANALYTICS WITH APACHE KAFKA AND SPARK A COMPREHENSIVE OVERVIEW

Authors

  • Asma Karachi University, Karachi, Pakistan.

DOI:

https://doi.org/10.71465/csb41

Keywords:

Real-time Analytics, Apache Kafka, Apache Spark, Stream Processing, Big Data, Event Streaming, Data Pipelines, Data Processing, Latency, Scalability

Abstract

The increasing need for real-time data processing has led to the evolution of scalable and distributed systems capable of handling large volumes of data with low latency. Apache Kafka and Apache Spark are two key technologies that have emerged as industry standards for real-time analytics. Apache Kafka, a distributed event streaming platform, and Apache Spark, an in-memory data processing engine, together form a robust architecture for processing real-time data streams. This paper explores the integration of Apache Kafka and Spark for real-time analytics, focusing on their architecture, use cases, challenges, and performance optimization techniques. The study aims to provide a comprehensive understanding of how these technologies work together to enable real-time data pipelines, offering insights into their advantages, limitations, and potential future advancements in big data analytics.

image

Downloads

Published

2020-06-30