Data Engineering

top of page

scalable data solutions

scalable data solutions

Scalable Data Solutions for Modern Businesses

Data Infrastructure

Claude Paugh

9 hours ago3 min read

data skipping

data skipping

Table Comparisons: Delta Lake, Apache Hudi ,and Apache Iceberg

Data Infrastructure

Claude Paugh

6 days ago5 min read

Unlocking the Potential of Scalable Data Engineering Practices

Unlocking the Potential of Scalable Data Engineering Practices

Unlocking the Potential of Scalable Data Engineering Practices

Data Infrastructure

Claude Paugh

Aug 253 min read

document database

document database

Comparing Couchbase and MongoDB: Insights on Features Performance and Scalability

Data Infrastructure

Claude Paugh

Aug 185 min read

joined dataframes

joined dataframes

Comparing Apache Spark and Dask DataFrames My Insights on Memory Usage Performance and Execution Methods

Claude Paugh

Aug 176 min read

Understanding Graph and Relational Databases: My Insights on Their Best Features and Use Cases

Understanding Graph and Relational Databases: My Insights on Their Best Features and Use Cases

Understanding Graph and Relational Databases: My Insights on Their Best Features and Use Cases

Data Architecture

Claude Paugh

Aug 174 min read

Database Design Solutions to Common Problems

Database Design Solutions to Common Problems

Database Design Solutions to Common Problems

Claude Paugh

Aug 113 min read

data center

data center

Scalable Data Engineering for IT Success

Data Engineering

Claude Paugh

Aug 74 min read

columnar formats

columnar formats

ORC vs Parquet which file format flexes harder in the data storage showdown

Data Infrastructure

Claude Paugh

Jul 244 min read

Users using SQL

Users using SQL

Datalake and Lakehouse: Comparison of Apache Kylin and Trino for Business Intelligence Analytics

Data Architecture

Claude Paugh

Jul 236 min read

spark architecture and catalyst optimizer

spark architecture and catalyst optimizer

Maximizing Scala Performance in Apache Spark Using the Catalyst Optimizer

Claude Paugh

May 196 min read

pandas analyzing data

pandas analyzing data

7 Easy Techniques to Detect Anomalies in Pandas for Data Analysis

Claude Paugh

May 144 min read

Apache Iceberg and Pandas Analytics: Part I

Apache Iceberg and Pandas Analytics: Part I

Apache Iceberg and Pandas Analytics: Part I

Data Infrastructure

Claude Paugh

May 76 min read

How to Leverage Python Dask for Scalable Data Processing and Analysis

How to Leverage Python Dask for Scalable Data Processing and Analysis

How to Leverage Python Dask for Scalable Data Processing and Analysis

Data Infrastructure

Claude Paugh

Apr 257 min read

Apache Spark Logon on screen

Apache Spark Logon on screen

Mastering Aggregations with Apache Spark DataFrames and Spark SQL in Scala, Python, and SQL

Data Infrastructure

Claude Paugh

Apr 244 min read

Shuffling

Shuffling

How I Optimized Apache Spark Jobs to Prevent Excessive Shuffling

Data Infrastructure

Claude Paugh

Apr 243 min read

Data Disks

Data Disks

How I Optimize Data Access for Apache Spark RDD

Data Engineering

Claude Paugh

Apr 243 min read

HDF5 data storage with graph

HDF5 data storage with graph

Understanding HDF5 The Versatile Data Format Explained with Examples

Claude Paugh

Apr 224 min read

ETF, Mutual Fund Holdings

ETF, Mutual Fund Holdings

ETF & Mutual Funds Portfolios: Infrastructure

Data Infrastructure

Claude Paugh

Apr 1912 min read

Spark

Spark

Apache Spark Best Practices: Optimize Your Data Processing

Data Engineering

Claude Paugh

Apr 164 min read

bottom of page