Credential expiry monitoring for Entra ID app registrations

An Entra ID tenant of any size accumulates app registrations: integrations, internal tools, vendor connectors, things someone set up in 2019 and left. Each has a client secret or a certificate, and each of those has an expiry date. When it passes, the integration stops. A ticket arrives a few days later, and the first hour goes into finding out who owns the app and what it was for.

In this tenant there were about a thousand of them.

Who owns this?

Entra ID does not require an application to have an owner, and a large share of these had none recorded, or an owner who had left years ago. A notice with nowhere to go is worthless, so half of the project was establishing one accountable contact per application, from the team that used it, the directory’s audit trail and whoever set it up, and writing it back to the registration so the next notice finds it without anyone asking.

Only what is in use

The second filter was just as important. Before notifying anyone I needed to know what was actually in use. A PowerShell script reads every app and every credential through Microsoft Graph, then matches each credential against roughly a thousand sign-in records from the last months. That separates “in use” from “forgotten”, and it matters: an expiring credential on a dead integration is a clean-up item, not an alarm.

Notifications go out only for credentials with recent sign-in activity; a stale secret beside an active certificate doesn’t raise an alarm. The same pass found around 60 credentials that had already expired, some by years. A companion script retires those, with a dry run by default and a backup of every object before it touches anything.

The notice, and who reads it

A Logic App writes to the app owner at 30 and 7 days before expiry. The owners are mostly product owners, not engineers. If the mail said “your credential keyId 3f2a… with permission Directory.Read.All and redirect URI https://… expires on the 14th”, they’d forward it to IT and the ticket would arrive anyway.

So the notice includes a short, plain-language summary: what this application appears to do, and what stops working if the credential lapses. That summary is generated by Azure OpenAI from the app’s configuration metadata: permission names resolved to something readable, redirect domains, sign-in audience, whether the credential shows recent use. No secrets or tokens are ever sent. The deterministic flags (does it involve user sign-in, does it expose an API) are computed first in PowerShell, and the model returns strict JSON with a confidence score, so low-confidence summaries get a human look before they go out.

Result

Owners now get a message they can act on without opening a ticket: renew, or confirm the app can be retired. Integrations stopped breaking on a Monday morning, and the clean-up list shrinks a little every month.