What MetaMiner Collects¶
Every collector MetaMiner runs against your tenant, what it reads, which Microsoft API family it uses, and how long it waits between completed cycles.
Collectors¶
The interval is the wait after a collector finishes a full cycle. A collector that has more pages or more work queued continues after about one second instead of waiting. All collectors of an environment run in parallel.
| Collector | What it reads | Microsoft API family | Interval |
|---|---|---|---|
| Workspaces | The list of every workspace in the tenant, including personal workspaces and their state (for example Deleted) | Power BI admin REST | 24 hours |
| Modified workspaces | Workspaces changed since the last check, to queue them for scanning | Power BI admin REST | 5 minutes |
| Workspace scan | Full metadata of new and changed workspaces: reports, dashboards and tiles, semantic models with tables, columns, measures and expressions, dataflows, data sources and their details, lineage, Fabric items, and who has access to the workspace and each item | Power BI admin REST (metadata scanning) | 2 minutes |
| Activity events | The tenant activity log (views, edits, exports, shares, refreshes and other user actions), read hour by hour | Power BI admin REST | 10 minutes |
| Refreshables | Refresh history and schedules of refreshable semantic models on capacities | Power BI admin REST | 10 minutes |
| Apps | Published apps and their contents | Power BI admin REST | 60 minutes |
| Capacities | Capacities with SKU, region, state and admins | Power BI admin REST | 3 hours |
| Gateways | On-premises and virtual network data gateways | Fabric REST | 3 hours |
| Connections | Gateway and cloud connections | Fabric REST | 1 hour |
| Fabric items | Every Fabric item in the tenant (lakehouses, warehouses, notebooks, pipelines, eventstreams, KQL items and more) | Fabric admin REST | 3 hours |
| Fabric item definitions | The definition of supported items (reports, semantic models, Dataflow Gen2, notebooks, data pipelines, eventstreams, KQL databases and querysets, eventhouses, Real-Time dashboards, Activator) in workspaces the application is a member of | Fabric REST | 3 hours (setting pbi.fabric.item.definitions.interval.minutes, default 180) |
| Dataflow definitions | The definition of Power BI dataflows | Power BI REST | 3 hours |
| Capacity Metrics | Capacity utilization from the Microsoft Fabric Capacity Metrics app: capacities, items, compute by item and operation per hour, CU detail, storage by workspace per hour | Power BI REST (DAX queries on the Capacity Metrics semantic model) | 60 minutes |
| Entra users | Users and their properties, kept current with change tracking | Microsoft Graph | 30 minutes |
| Entra service principals | Service principals | Microsoft Graph | 30 minutes |
| Entra applications | Application registrations | Microsoft Graph | 30 minutes |
| Entra groups | Groups | Microsoft Graph | 30 minutes |
| Entra group members | Group membership | Microsoft Graph | 30 minutes |
| Subscribed SKUs | License products the tenant owns and their counts | Microsoft Graph | 24 hours |
| User licenses | Licenses assigned to each user | Microsoft Graph | 30 minutes |
| Group licenses | Licenses assigned through groups | Microsoft Graph | 30 minutes |
Personal and deleted workspaces
The admin APIs return personal workspaces ("My workspace") as well as shared ones. The Power BI Explorer groups them under Personal Workspaces. Workspaces that Microsoft reports as Deleted stay visible so that Fabric administrators can restore them.
How far back the first collection reads¶
| Data | First collection reads | Later collections |
|---|---|---|
| Activity events | The last 28 days (Microsoft keeps about 30 days) | From the last hour read, with a one-hour overlap |
| Modified workspaces | Changes in the last 30 days | From the last check, with a one-hour overlap |
| Capacity Metrics | The last 14 days | Hour by hour from the last point read |
| Entra directory | A full read | Only changes, using Microsoft Graph change tracking |
What each collector needs¶
| Collectors | Needs |
|---|---|
| Power BI admin REST collectors and Fabric items | The application in a security group that is allowed to use read-only admin APIs; the two "enhance admin API responses" settings for full scan detail |
| Gateways, connections | The tenant setting that lets service principals use Fabric APIs |
| Fabric item definitions, dataflow definitions | Fabric API access plus membership of the application in the workspaces concerned |
| Capacity Metrics | The Capacity Metrics app installed, its semantic model ID entered in MetaMiner, and the application as Contributor or Admin of that workspace |
| Microsoft Graph collectors | Graph application permissions with admin consent |
Details: Microsoft Permissions Overview.
TODO: confirm the per-item-type definition list against the release
Source: the item type registry shipped with the release. Current list: Report, SemanticModel, Dataflow (Gen2), Eventstream, KQLDatabase, Eventhouse, KQLQueryset, DataPipeline, Notebook, Reflex (Activator), KQLDashboard. Lakehouse, Warehouse and SQL endpoint are inventory only.
What MetaMiner does not collect¶
- Data rows inside semantic models, reports, lakehouses or warehouses.
- Report visuals as images.
- Credentials stored in connections or gateways.