Astera

JUNE 6, 2023

Technologies like ETL, batch processing, real-time streaming, and data warehouses are used. They are designed for streaming data sources, such as sensors, logs, or social media feeds. Assess whether real-time streaming or batch processing is suitable for your pipeline needs.

Data Warehouse 52

Data Warehouse Monitoring Real-time Data Documentation 52

Whizlabs

APRIL 4, 2021

AWS Kinesis is one of the most popular data streaming and analytics solutions in the present. Data streaming is one of the most important requirements for businesses in the present times for various reasons. Amazon Kinesis is a real-time, fully managed, and highly scalable cloud service for streaming large volumes of data on AWS.

Real-time Data 64

Real-time Data Data Analytics Data Analytics Monitoring 64

Astera

JUNE 13, 2023

Real-time Pipelines Real-time pipelines handle data streams in real time, enabling instant analysis and decision-making. These pipelines are designed to process data as it arrives, ensuring immediate availability of insights and timely actions based on streaming data. ETL processes are often used for batch processing scenarios.

E-Commerce 52

E-Commerce Data Quality Real-time Data Data Warehouse 52

Astera

JUNE 13, 2023

Real-time Data Pipeline: Handles data in a streaming fashion, essential for time-sensitive applications and immediate insights. Some popular ones include Astera Data Stack Apache Airflow Apache Kafka Apache NiFi AWS Glue. Cloud Data Pipeline: Leverages cloud infrastructure for seamless data integration and scalable processing.

Data Warehouse 52

Data Warehouse Data Quality Real-time Data Visualization 52

Astera

JUNE 13, 2023

Real-time Data Pipeline: Handles data in a streaming fashion, essential for time-sensitive applications and immediate insights. Some popular ones include Astera Data Stack Apache Airflow Apache Kafka Apache NiFi AWS Glue. Cloud Data Pipeline: Leverages cloud infrastructure for seamless data integration and scalable processing.

Data Warehouse 52

Data Warehouse Data Quality Real-time Data Visualization 52

Astera

JUNE 13, 2023

Real-time Data Pipeline: Handles data in a streaming fashion, essential for time-sensitive applications and immediate insights. Some popular ones include Astera Data Stack Apache Airflow Apache Kafka Apache NiFi AWS Glue. Cloud Data Pipeline: Leverages cloud infrastructure for seamless data integration and scalable processing.

Data Warehouse 52

Data Warehouse Data Quality Real-time Data Visualization 52

Astera

APRIL 24, 2024

Real-Time Data Streaming Tools : These tools ingest data continuously as it appears, making it available for immediate analysis. Apache Kafka Apache Kafka is an open-source distributed platform suitable for real-time data ingestion. Pros It supports low latency for real-time data streaming.

Data Warehouse 52

Data Warehouse Real-time Data Documentation Data Quality 52

Cprime

SEPTEMBER 3, 2021

Linear computer science theorists will agree with the line of thinking that as more data streams around the world, the difficulty in accessing resources on the internet will increase likewise. However, with platforms like Apache Kafka, the opposite is true. Key Features of Apache Kafka. Getting Started With Apache Kafka.

Banking 52

Banking Big Data 52

Dataversity

MAY 2, 2023

Data stream processing is rapidly emerging as a critical technology for modernizing enterprise applications and improving real-time data analysis for data-driven applications. Traditionally, […] The post Leveraging Data Stream Processing to Improve Real-Time Data Analysis appeared first on DATAVERSITY.

Real-time Data 130

Real-time Data Data Architecture 130

InfoQ Articles

MAY 29, 2023

Cloudflare uses Kafka clusters to decouple microservices and communicate the creation, change or deletion of various resources via protobuf, a common data format in a fault-tolerant manner.

Documentation 101

Documentation 101

Whizlabs

APRIL 10, 2021

The major purpose of AWS Kinesis is to obtain, store and process various data from different distributed streams. Some of the commonly distributed streams are social media feeds, event logs, and others. Moreover, creating a new stream, setting the requirements, and data streaming can be done at a faster pace with AWS Kinesis.

Real-time Data 72

Real-time Data Big Data Digital Data Analytics 72

InfoQ Articles

FEBRUARY 6, 2023

Apache Kafka has integrations with most of the languages used these days, but in this article series, we cover its integration with Java. In this series, we also discuss how to provision, configure and secure an Apache Kafka cluster on a Kubernetes cluster. By Alex Soto

86

86

InfoQ Articles

AUGUST 15, 2022

It’s common in microservices to write data in two places, a database and send the content to another microservice. One approach to tackle this problem is dual writes, but you may lose data because of concurrent writes.

110

110

InfoQ Articles

DECEMBER 30, 2022

Deploying an Apache Kafka cluster to Kubernetes is easy if you use Strimzi, but that’s only the first step; you need to secure the communication between Kafka and the consumers and producers, provide RBAC to access topics, spread the secrets correctly to Kafka Connect components and all using a Kubernetes GitOps way.

71

71

InfoQ Articles

SEPTEMBER 30, 2022

At QCon Plus 2021 last November, Sid Anand, Chief Architect at Datazoom and PMC Member at Apache Airflow, presented on building high-fidelity nearline data streams as a service within a lean team. In this talk, Sid provides a master class on building high-fidelity data streams from the ground up. By Sid Anand.

52

52

Sisense

NOVEMBER 30, 2020

When an athlete views performance data, an activity or a live data feed on Strava, this triggers Strava’s Kafka event streaming to log the data. After streaming to Redshift , the data can be queried and piped to app dashboards. Strava features are instrumented through its logging infrastructure. Rosetta Stone.

Embedded Analytics 98

Embedded Analytics Predictive Analytics Data Warehouse Innovation 98

The Data Administration Newsletter

FEBRUARY 15, 2022

Most large technology businesses collect data from their consumers in a variety of methods, and the majority of the time, this data is in its raw form. However, when data is presented in an understandable and accessible style, it may assist and drive business requirements. The task is to process the data and, if required, […].

Real-time Data 52

Real-time Data Data Architecture Big Data Business Intelligence 52

Actian

NOVEMBER 25, 2019

Modern Edge Intelligence needs support for in-line analytics based on more than just the locally sourced data streams. Going forward, there will be multiple data streams and data types with baseline patterns that are referenced, correlated, with machine learning algorithms applied.

Artificial Intelligence 40

Artificial Intelligence Data Warehouse Data Management Visualization 40

Data Pine

SEPTEMBER 4, 2023

On Facebook, Instagram, or Twitter, we scroll through streams and streams of content. Users scroll, not click, to see charts change and follow the investigation step by step. The scroll action is the new users’ favorite, which is particularly fit for the mobile user.

Data Visualization 105

Data Visualization Visualization IBM cost IBM Support 105