Skip to content

Collection Problems

Fix problems with MetaMiner collecting Power BI, Fabric and Entra data: throttling, missing workspaces, failed scans, stale items and empty Entra or capacity data.

How collection works

For each enabled Power BI environment, the MetaMiner Server (MMS) runs a set of independent collectors. Each collector calls one Microsoft API on its own cadence and writes the results to the MetaMiner Database (MMDB). Every 10 minutes the MMS rebuilds the view the MetaMiner Client (MMC) shows from the database. A failing collector does not stop the others. See Collection Processors for the list.

Where to look What it tells you
MMA environment list Whether the environment is enabled and validated
Collection Status Freshness per environment
server\logs\server.log Collector errors, with the collector name (for example WORKSPACE_SCAN)
server\logs\statereport.log An hourly state report with the status of each collector

Environment changes need a restart

The MMS reads the Power BI environment settings when it starts. After you change credentials, the endpoint set or the Capacity Metrics selection in the MMA, restart the MetaMiner Server.

Throttling (HTTP 429)

Collection slows down or pauses; the logs mention 429 or "too many requests"

Symptom. A collector stops advancing for a while; server.log shows HTTP 429 for one collector; data on the dashboards falls behind and then catches up.

Cause. Microsoft limits how often a tenant may call its admin APIs, especially the workspace scanner and the activity log. The limits apply to the whole tenant, so other tools that call the same APIs (scripts, other governance products, a second MetaMiner server on the same tenant) use the same allowance.

What MetaMiner does.

  • When Microsoft answers 429, only that collector pauses. It waits as long as Microsoft asks, or 20 minutes when Microsoft gives no wait time (pbi.api.timeout.on.activity.limit.reached). It then resumes where it stopped; nothing is lost.
  • The Fabric item definition collector honours Microsoft's wait time and starts no new requests until it ends.
  • MetaMiner also counts its own calls per hour to the workspace and scanner APIs and holds back before reaching Microsoft's published limits.
  • After other errors a collector retries after 1, 5, 10, 30 and 60 seconds, then every 5 minutes, until a call succeeds.

Fix. Usually none: collection recovers by itself. If throttling is constant, find other tools calling the Power BI admin APIs for the same tenant and schedule them apart. Configure each tenant in one MetaMiner environment only.

What to send support. server.log and statereport.log covering a few hours, and the names of other tools calling the admin APIs.

Missing workspaces or items

A workspace, report or dataset is missing in the MMC or the dashboards

Cause and timing. MetaMiner finds workspaces two ways: the full workspace list once a day, and the list of workspaces Microsoft reports as modified, every 5 minutes. Found workspaces are then scanned in batches of up to 100, one batch after the other. On a large tenant the first complete scan can take hours. A new item appears after its workspace's next scan plus the next 10-minute view rebuild.

Fix.

  1. Check that the environment is enabled and that validation passes, in particular Service principals can access read-only admin APIs and PBI Workspaces Scan API.
  2. Check the tenant settings for detailed metadata and expressions in admin API responses. Without them Microsoft returns workspaces with fewer details. See Tenant Settings.
  3. Wait for the next cycle: a few minutes for a modified workspace, up to a day for a workspace that has not changed since it was created.
  4. Search server.log for errors from GET_WORKSPACES, GET_MODIFIED_WORKSPACES or WORKSPACE_SCAN.

What to send support. The workspace ID, when it was created or changed, and a log bundle.

TODO: confirm whether personal workspaces (My workspace) are collected

Source to check: the workspace list and scan filters. State the answer here and in the FAQ.

Fabric item definitions (report and semantic model definitions, notebooks, pipelines) are missing

Cause. Microsoft returns an item's definition only to a principal that is a member of the workspace. MetaMiner downloads definitions only in workspaces where its service principal, or a group it belongs to, has the Admin, Member or Contributor role, and for reports only when it may write to the report's semantic model. Definitions are refreshed every 3 hours, for items changed since the last download.

Fix. Add the service principal's security group to the workspaces whose definitions you need. See Workspace Access.

Scan failures

Some workspaces never get their details, or the logs show scan errors

Cause. A scan request can be refused, fail inside Microsoft, or return a result MetaMiner cannot read. MetaMiner records the failed workspaces and retries them automatically in later batches. A scan that does not complete after many status checks is marked failed so the next batch can run.

Fix.

  1. Validate the environment to rule out permission changes.
  2. Look in server.log for WORKSPACE_SCAN errors and the workspace IDs.
  3. Very large workspaces produce large scan results. MetaMiner processes them in parts, but they need heap; see Performance and Memory.

When MetaMiner cannot read a scan result, it keeps a copy for diagnosis under the server's Java temp folder in mm-pbi-dumps (at most 4 GB, 10 files, 30 days; property pbi.ws.scan.dump.budget.gb).

What to send support. The workspace IDs, the matching server.log lines, and ask support before sending a dump file; it can contain metadata of the whole workspace.

Stale items

An item deleted or renamed in Power BI still shows its old state

Cause. A change shows after Microsoft reports the workspace as modified (checked every 5 minutes), the workspace is scanned again, and the MMS rebuilds the MMC view (every 10 minutes). Press ++f5++ in the MMC to load the newest view.

Fix. Wait 20-30 minutes and refresh. If a deleted item is still shown after the workspace was scanned again, restart the MetaMiner Server: the restart rebuilds the view from the database. Removed items are kept in an archive for 180 days (pbi.deleted.data.retention.days) before they are purged.

What to send support. The item ID, when it was changed, and a log bundle.

Entra (Graph) data is empty

The Entra tab, licence figures or owner status are empty

Cause. The Microsoft Graph collectors cannot read users, groups, group members, applications, service principals, subscribed SKUs or licence assignments. Usually a missing Graph application permission or missing admin consent.

Fix.

  1. Validate the environment and fix any failed Graph role (User.Read.All, Group.Read.All, GroupMember.Read.All, Application.Read.All, LicenseAssignment.Read.All, or Directory.Read.All). See Validation Errors.
  2. Wait for the collectors: the first run reads everything; afterwards they read changes every 30 minutes. Subscribed SKUs are read once a day.
  3. Search server.log for errors from the GRAPH_* collectors.

What to send support. The validation results and the GRAPH_* lines from server.log.

Capacity metrics are empty

The Capacity Metrics dashboard or capacity data in the MMC is empty

Cause. One of:

  • No Capacity Metrics semantic model is configured for the environment, or Ignore FCM at this time is ticked.
  • The service principal is not Contributor or Admin on the workspace hosting the Capacity Metrics app.
  • The capacities you want are not ticked in the environment's capacity selection, or they are trial capacities (collected only with pbi.fcm.trial.capacities.enabled=true).
  • The Capacity Metrics app has not been refreshed, or its version uses a layout MetaMiner does not support.

Fix. Validate the environment and fix the Fabric Capacity Metrics Data step. See Capacity Metrics Datasets. The collector runs hourly and first reaches back 14 days. Restart the MetaMiner Server after changing the selection.

What to send support. The validation results and FCM lines from server.log.

Activity data is behind

The dashboards' LIVE indicator shows an old time, or recent activity is missing

Cause. The activity collector reads the Power BI activity log every 10 minutes in one-hour windows. Microsoft publishes activity events with a delay, so the most recent hour can be incomplete. On first collection MetaMiner reaches back about 28 days; older activity is not available from Microsoft.

Fix. Check the ACTIVITY_EVENTS lines in server.log for errors or throttling, and the rollup jobs in Data Freshness.

Network errors

Collectors fail with connection resets, timeouts or TLS errors

Cause. The MMS cannot complete HTTPS connections to the Microsoft endpoints: a firewall, proxy or TLS inspection device interferes. The affected collectors retry with growing waits.

Fix. Allow the endpoints in Ports and Endpoints from the MMS machine and exempt them from TLS inspection.

What to send support. The error lines from server.log and the result of Test-NetConnection api.powerbi.com -Port 443 on the MMS machine.

TODO: document outbound proxy support for the Power BI collectors

Confirm how the Power BI, Fabric and Graph calls use an outbound HTTP proxy (system proxy properties, authenticating proxies) and document the supported configuration.

Next steps