Lineage
Which tables feed which, and what breaks if you delete one. Read from Google's Data Lineage API.
The Lineage page (/lineage) shows which tables feed which, and what depends on each table. Use it before you delete or change a table.
Lineage comes only from Google's Data Lineage API (Dataplex). SlashPeak does not guess lineage from query jobs.
Turn it on
- Enable the Data Lineage API in each GCP project. See Enable Google APIs.
- Grant the scanner
datalineage.locations.searchLinks(part ofroles/datalineage.viewer). See IAM permissions. - Run a scan.
If lineage is off or not granted, the page shows these three steps and the last error from Google.
Google keeps links for 30 days
Google records a link only when a job writes one table from another, after the API is on. It keeps each link for 30 days. If the page says "No lineage links right now", run such a job, then run a scan.
The table list
Each row is one table:
| Column | Meaning |
|---|---|
| Storage / mo | Monthly storage cost |
| Upstream | Tables that feed this one |
| Downstream | Tables that read from this one |
| Reports | Reports and dashboards that read it, when Google records them |
| Last queried | The last query that read it in the scan window |
| Verdict | Dead end, Hot or Normal |
Verdicts:
- Dead end: nothing reads from this table in lineage, and no query used it in the scan window. It may be safe to delete.
- Hot: many tables or reports depend on it. Change it with care.
- Normal: some downstream use.
Filter by project, or search for a table.
One table
Click a table to see:
- If you delete this table: how many tables, reports and other things downstream depend on it.
- The upstream and downstream graph.
- Open in Data: the table's page on Data.
Looker and other BI tools
SlashPeak does not log in to Looker. To see the cost of Looker queries, open Usage → Query and filter Job source = Looker.
If the page is empty
- Enable the Data Lineage API and grant the permission above.
- Run a job that writes a table from another table, then scan again.
- Check that the project is not excluded on the connection.
- If a project has very many links, the page says it stored only part of them. Counts for that project may be incomplete.