Data Engineering | Perardua Consultinghttps://www.perarduaconsulting.com/blog/tags/data-engineering https://static.wixstatic.com/media/5584dc_9de4690541404b9b980acea3d27a8cba~mv2.jpg

top of page

scalable data solutions

scalable data solutions

Scalable Data Solutions for Modern Businesses

Data Infrastructure

Claude Paugh

9 hours ago3 min read

Best Practices for Utilizing the Medallion Method in ETL and ELT for Data Lakes vs Lakehouses

Best Practices for Utilizing the Medallion Method in ETL and ELT for Data Lakes vs Lakehouses

Best Practices for Utilizing the Medallion Method in ETL and ELT for Data Lakes vs Lakehouses

Data Architecture

Claude Paugh

5 days ago5 min read

snowflake

snowflake

Delta Lake vs Snowflake Lakehouse: Analyzing Ecosystems, Large Datasets, and Query Optimization

Data Infrastructure

Claude Paugh

6 days ago4 min read

Unlocking the Potential of Scalable Data Engineering Practices

Unlocking the Potential of Scalable Data Engineering Practices

Unlocking the Potential of Scalable Data Engineering Practices

Data Infrastructure

Claude Paugh

Aug 253 min read

document database

document database

Comparing Couchbase and MongoDB: Insights on Features Performance and Scalability

Data Infrastructure

Claude Paugh

Aug 185 min read

table

table

Understanding the Distinctions Between First to Fifth Normal Forms in Data Modeling

Claude Paugh

Aug 113 min read

data center

data center

Scalable Data Engineering for IT Success

Data Engineering

Claude Paugh

Aug 74 min read

Data Lake and Data

Data Lake and Data

Comparing Key Differences Between Databricks and Snowflake for Your Data Needs

Claude Paugh

Aug 65 min read

columnar formats

columnar formats

ORC vs Parquet which file format flexes harder in the data storage showdown

Data Infrastructure

Claude Paugh

Jul 244 min read

cloud

cloud

Comparing Apache Parquet, ORC, and JSON File Formats for Your Data Processing

Data Infrastructure

Claude Paugh

Jul 84 min read

Spark, Hive, & Hadoop

Spark, Hive, & Hadoop

Apache Iceberg, Hadoop, & Hive: Open your Datalake (Lakehouse) -> Part II

Data Infrastructure

Claude Paugh

Jun 247 min read

Apache Iceberg and Pandas Analytics: Part I

Apache Iceberg and Pandas Analytics: Part I

Apache Iceberg and Pandas Analytics: Part I

Data Infrastructure

Claude Paugh

May 76 min read

Data Vault Hubs, Satellites, and Links

Data Vault Hubs, Satellites, and Links

Data Vault Modeling Design Uses

Data Infrastructure

Claude Paugh

May 29 min read

How to Leverage Python Dask for Scalable Data Processing and Analysis

How to Leverage Python Dask for Scalable Data Processing and Analysis

How to Leverage Python Dask for Scalable Data Processing and Analysis

Data Infrastructure

Claude Paugh

Apr 257 min read

Apache Spark Logon on screen

Apache Spark Logon on screen

Mastering Aggregations with Apache Spark DataFrames and Spark SQL in Scala, Python, and SQL

Data Infrastructure

Claude Paugh

Apr 244 min read

Shuffling

Shuffling

How I Optimized Apache Spark Jobs to Prevent Excessive Shuffling

Data Infrastructure

Claude Paugh

Apr 243 min read

Data Disks

Data Disks

How I Optimize Data Access for Apache Spark RDD

Data Engineering

Claude Paugh

Apr 243 min read

HDF5 data storage with graph

HDF5 data storage with graph

Understanding HDF5 The Versatile Data Format Explained with Examples

Claude Paugh

Apr 224 min read

Apache Iceberg and HDF5

Apache Iceberg and HDF5

Exploring Apache Iceberg and HDF5 Use Cases in Modern Data Management

Claude Paugh

Apr 224 min read

Unlocking the Potential of Apache Iceberg in Cloud-Based Data Engineering Strategies

Unlocking the Potential of Apache Iceberg in Cloud-Based Data Engineering Strategies

Unlocking the Potential of Apache Iceberg in Cloud-Based Data Engineering Strategies

Claude Paugh

Apr 224 min read

bottom of page