Further Tests & Elementary¶

Group 2 - Part 2

So far you've been testing models you know well. This topic is about stepping back and asking a project-wide question: by the standards of a well-run dbt project, where are the gaps? You'll use dbt-project-evaluator to answer that question automatically, then decide what's actually worth fixing.

Exercise¶

mediapulse_base was deliberately built with partial coverage - some models are fully tested and documented, others are not. That's realistic: most production dbt projects look like this. Your job is to find the gaps with tooling rather than by eyeballing every YAML file.

Step 1 - Install the evaluator¶

Step complete

Check packages.yml in mediapulse_base - dbt-project-evaluator isn't there yet. Add it and install it.

Hint: Which package

It's dbt-labs/dbt_project_evaluator. Check the dbt-project-evaluator documentation for the current version range to pin, then run dbt deps.

Step 2 - Run it and look at the output¶

Step complete

The package ships as a set of models, not a CLI tool - you build it like any other package, then query its result tables.

Hint: Building the evaluator

Select just the package's models with dbt build --select package:dbt_project_evaluator. Its models write their findings into tables you can query directly - start with fct_missing_primary_key_tests and fct_undocumented_models.

Step 3 - Pick three real findings and read them critically¶

Step complete

At least one of them should be about stg_ads__campaigns, stg_ads__impressions, or stg_ads__spend. Look at what the evaluator flags for these models and compare it to what you already know about how they're built.

Hint: A finding you should recognise

These three staging models read straight from mediapulse_raw.ads.* rather than through a source() definition - that's already called out in their own YAML descriptions. Check whether fct_direct_join_to_source (or the absence of a source definition entirely) surfaces this, and whether the evaluator's framing of the problem matches your own judgement of its severity.

Step 4 - Triage what you found¶

Step complete

For each of your three findings, decide: must-fix, should-fix, or acceptable-by-design. Write one sentence of justification for each - not just what the rule says, but whether breaking it is actually costing anyone anything right now.

Step 5 - Fix at least one¶

Step complete

Pick your highest-priority "must-fix" and resolve it. Re-run the evaluator selection and confirm the specific violation is gone.

Deliverable: Your triage table (finding, verdict, one-line justification) plus a before/after note on the one violation you fixed.

Extension¶

mediapulse_analytics is a second project in the same mesh, and it's structurally different from mediapulse_base - some of its marts (fct_content_performance, fct_ad_revenue) depend on staging models that live in the other project, while its crm and streamview_legacy domains are entirely self-contained, with their own sources, staging, and marts.

Step 1 - Run the evaluator on `mediapulse_analytics`¶

Step complete

Add dbt-labs/dbt_project_evaluator to mediapulse_analytics/packages.yml, run dbt deps, and build it here too. Expect a different shape of findings than mediapulse_base - this project has no intermediate layer at all, for one.

Step 2 - Fix one real gap here as well¶

Step complete

Pick one genuine finding from mediapulse_analytics's results - for example a missing test on a column in the crm or streamview_legacy domain's YAML - and fix it the same way you did in the Exercise. Re-run the evaluator selection to confirm it's gone.

Step 3 - Test the "staging layer" assumption¶

Step complete

dbt-project-evaluator's modeling rules generally assume every mart traces back to a staging model in the same project. fct_content_performance and fct_ad_revenue don't - their staging models live upstream in mediapulse_base, reached via the mediapulse_base entry in dependencies.yml. Does the evaluator understand this, or does it flag these marts as if they were skipping staging entirely? Read the dbt-project-evaluator documentation for how (or whether) it accounts for cross-project dependencies.

Step 4 - Decide what "good" looks like here¶

Step complete

Write up your own answer to: should mediapulse_analytics's local domains (crm, streamview_legacy) be held to the identical structural rules as a fully standalone project, given that the project as a whole is intentionally a mix of "owns its own layer" and "consumes another project's layer"? Where would you override or suppress a rule versus actually fix the model?

Step 5 - Propose a rule override¶

Step complete

If you found a rule that produces a false positive for this project's architecture, look up how dbt-project-evaluator lets you disable or reconfigure individual rules at the project level, and describe (in words, not code) exactly what you'd change and why.

Deliverable: The fix you made in step 2, plus a short write-up comparing the evaluator's usefulness on a "normal" single-project layout (mediapulse_base) versus a mesh-consuming project with mixed local/cross-project domains (mediapulse_analytics) - where did the tool's default assumptions hold up, and where did they need judgement?

Done?

You've run a real audit tool across both projects, fixed genuine violations, and learned where an automated evaluator's assumptions need a human's judgement.

Now head to dbt Mesh to go deeper into groups and how mesh access decisions actually get made.