Data Pipelines and BI - BigQuery, dbt, Power BI

A client put it this way on a finished job: “He was extremely proficient, fast and responsive. Highly recommended!” - Tobias B., founder of Split My Fare, after moving Firebase data into Google Cloud and BigQuery.

Two decades of this, from Oracle OBIEE and ODI early on through to BigQuery, dbt and Airflow now. The tools change; the failure does not. Data lands somewhere nobody can query, or it lands somewhere queryable with no agreed definitions - and either way the dashboards get quietly ignored.

What's included in the package

Ingestion from where the data actually is

Firebase, Google Analytics, application databases, third-party APIs, flat files. Batch or streaming.

A warehouse with a modelled layer

BigQuery or Postgres, with dbt models so business logic lives in version control instead of inside a dashboard.

Scheduling and alerting

Airflow or Cloud Scheduler, and a message when a load fails - silence is not success.

Dashboards people open twice

Power BI, Looker Studio or Superset, built around the questions the business actually asks.

Documented definitions

What counts as an active customer, written down, so two dashboards cannot disagree.

How the installation goes

Find the questions first

Not the tables. A warehouse built without knowing what will be asked of it is a very expensive copy of your database.

Land the raw data

Ingest untransformed and keep it. Every pipeline needs somewhere to go back to when a definition changes.

Model it

dbt models with tests, so a broken assumption fails the build instead of quietly changing a number on a dashboard.

Build the dashboard last

It is the easiest part, and the part everyone starts with.

Frequently asked questions

Have you done this before? ▾

Repeatedly, with public feedback. Firebase and Google Analytics into BigQuery at 5.0 stars with two follow-on contracts, a dbt customer-modelling engagement of 74 hours at 5.0, a real-time NetFlow/IPFIX pipeline into dashboards, Airbyte on GKE, plus Power BI suites, Superset and Looker Studio.

We already have dashboards nobody trusts. Can that be fixed? ▾

Usually. Distrust nearly always traces to definitions living in several places and disagreeing. The fix is a modelled layer where the definition is written once and tested.

How much will BigQuery cost us? ▾

Less than people fear if tables are partitioned and clustered properly, and alarming if they are not. I will size it against your actual volumes before you commit.

Other things I build

AI Integration and RAG - Answers From Your Own Documents

Retrieval-augmented AI that answers from your documents and cites the page it used, or says it does not know. Built and run in production, not a demo.

Automation and Integrations - WhatsApp, Stripe, QuickBooks, Airtable

Connect the systems your business already runs and automate the work in between. WhatsApp Business API, Stripe, Twilio, QuickBooks, Airtable and Google Workspace.

Software Support and Maintenance - Take Over an Existing System

Inherit, stabilise and keep running software someone else built. Bug fixes, upgrades, cloud costs, and an honest assessment of what you actually have.

The work behind these is public: the AI product, its live demo, the cited answer pages it produces, and the research paper measuring it.