My GCP Pipeline Was Running Fine… But Doing Nothing (Service Account Lesson)
A real debugging story on how missing IAM permissions in a service account silently broke my Cloud Function, Dataflow trigger, and BigQuery pipeline.
May 2, 20264 min read3
Command Palette
Search for a command to run...
#data-engineering
Articles tagged with #data-engineering
7 Skills That Make Data Engineering Feel Less Hard
Most data engineering problems are not what they seem and it took me time to realize what actually matters
Apr 25, 20263 min read1
Dataflow Job Not Starting? Debugging a Job Name Collision in GCP
Debugging a silent Dataflow failure caused by duplicate job names in a GCS-triggered Cloud Function pipeline
Apr 18, 20264 min read
The Small metadata.json File That Changed How I Work With Image Datasets
A short story about how adding a simple metadata.json file changed the way I organized and analyzed an image dataset collected from multiple cameras.
Apr 11, 20264 min read
When a Cloud Function Keeps Failing and the Code Isn’t the Problem
A small IAM configuration mistake that breaks functions interacting with Storage, Firestore or other services.
Apr 4, 20265 min read
BigQuery Project and Dataset Organization Best Practices
Best practices for organizing BigQuery projects, datasets, and tables in real data platforms
Mar 31, 20264 min read