Report-Only Shows 'Not Applied': How to Test a Block Legacy Authentication Policy with Sign-In Logs
By Mirko Peters ·
A report-only Conditional Access policy that only shows Not applied may be good news. Find real legacy authentication in the sign-in logs and read the results before you enable the block.
A team creates a Conditional Access policy that blocks legacy authentication and leaves it in report-only mode. They open the Conditional Access insights and see only one result, reportOnlyNotApplied. Is the policy broken? Can it be turned on safely?
Key points
- Report-only mode does not enforce anything and does not generate traffic. A policy only shows a result when a sign-in matches its conditions.
- Learn says to look for legacy authentication in both the Sign-in logs and the User sign-ins (non-interactive) tab, filtered by Client App.
- Report-only: Not applied means not all conditions were met. For a block-legacy policy, no Failure results suggests no legacy authentication in the period you checked.
- Exclude emergency access accounts, and at least one account that must keep legacy authentication, so you cannot lock yourself out.
This question was asked in a Microsoft Q&A thread in March 2026. A Microsoft moderator answered, the answer was accepted, and the person who asked thanked them. The first reply used a filter name that did not exist in the asker's portal, and a later reply corrected it. I checked the steps against the current Microsoft Learn pages. This is not an official Microsoft procedure.
What the symptom really tells you
Learn lists four report-only results:
| Result | Learn's meaning | |---|---| | Report-only: Success | All conditions were met, and the required grant and session controls were satisfied | | Report-only: Failure | All conditions were met, but not all required grant or session controls were satisfied | | Report-only: User action required | All conditions were met, but the user would have needed to act, for example to complete MFA. Users are not prompted in report-only mode | | Report-only: Not applied | Not all conditions were met, for example the user is excluded |
The asker's value reportOnlyNotApplied corresponds to the last row. For a policy that targets legacy clients, "not applied" means the sign-ins in that view did not match the policy. The moderator in the thread said the same and added that report-only mode does not simulate traffic.
Learn also notes that report-only mode does not enforce grant or session controls. Nobody is blocked or prompted.
Step 1: Check how the policy is set up
Learn describes the policy this way. You need at least the Conditional Access Administrator role.
- In the Microsoft Entra admin center, go to Entra ID > Conditional Access > Policies > New policy.
- Under Users or workload identities, include All users. Exclude your emergency access accounts.
- Under Target resources, include All resources (formerly 'All cloud apps').
- Under Conditions > Client apps, set Configure to Yes and check only Exchange ActiveSync clients and Other clients.
- Under Grant, select Block access.
- Set Enable policy to Report-only and select Create.
Learn also tells you to exclude at least one account that must keep legacy authentication, so you do not lock yourself out, and to remember that service principals are not covered by user-scoped policies.
Step 2: Look for real legacy sign-ins in the logs
Learn says to use at least the Reports Reader role and do this:
- Go to Entra ID > Monitoring & health > Sign-in logs.
- Add the Client App column through Columns.
- Select Add filters > Client App and choose the legacy authentication protocols.
- Repeat on the User sign-ins (non-interactive) tab.
The Learn page stresses checking both places. The sign-in log has four types: interactive user, non-interactive user, service principal and managed identity sign-ins. The older sign-in log experience shows only interactive sign-ins.
This is where the thread got stuck. The asker could not find a "Legacy authentication clients" option, because the portal listed Other clients. The moderator's corrected advice was to filter Client app on Other clients in the User sign-ins (non-interactive) logs. Learn names Other clients and Exchange ActiveSync clients as the legacy options in the policy, but it does not say in the pages I used that every legacy protocol appears as Other clients. So check both, and open a sample sign-in to read its Client App value in the sign-in details.
Learn also points to the Sign-ins using legacy authentication workbook for triage.
Step 3: Add the Conditional Access result to the view
In the thread, the accepted answer told the asker to add the Conditional Access column to the non-interactive view and read the result for the policy. To check one event in detail, open the sign-in and select the Conditional Access tab. Learn's troubleshooting page says this tab shows the policies that affected the sign-in. Your portal may also show a separate report-only view there, which the Learn pages I used do not describe.
How to read what you see:
- Report-only: Failure on a sign-in that used a legacy client. For a policy with Block access, the grant control cannot be met, so this is what you would expect for sign-ins that would be blocked. The moderator in the thread described it the same way. Learn's general definition of Failure fits, but its pages do not spell out this exact case.
- No entries, or only Not applied. Legacy authentication was probably not in use in the period you checked. The moderator said that enabling the policy should then not affect users.
Check a long enough period. A monthly job or a rarely used scanner may appear only once a month. Learn does not state the log retention in the pages I used, so confirm your own retention.
Step 4: Use the aggregated views
If you do not want to open events one by one, which the asker asked about, Learn describes two options:
- Policy impact, available to admins with at least the Security Reader role. It shows the impact on interactive sign-ins for the past 24 hours, 7 days or 1 month. Because legacy protocols mostly show up as non-interactive sign-ins, treat it as a supplement to the log filter, not a replacement.
- The Insights and Reporting workbook. Learn says it can compare report-only and enforced results, and that it needs Microsoft Entra ID P1 and a Log Analytics workspace that receives sign-in logs.
A Conditional Access What If tool exists, and Learn links to it from its troubleshooting page, but I did not verify its steps for this article.
Step 5: Fix what you find
For each legacy sign-in you find, identify the app or device and its owner. Move it to modern authentication, or replace a service account used in scripts with a managed identity, as Learn suggests. Learn's policy steps exclude accounts that must keep legacy authentication, so keep that exclusion list as short as you can.
Step 6: Turn it on in stages
Learn says that after you have validated the impact, you move the Enable policy toggle from Report-only to On. Once the policy is on, Learn's list of Conditional Access error codes includes AADSTS53003, shown as BlockedByConditionalAccess, which is the code you would look for on blocked sign-ins. Keep the emergency access exclusion in place.
One caution for other report-only policies
Learn warns that report-only policies that require a compliant device can show certificate prompts to users on macOS, iOS and Android, and those prompts can repeat until the device is compliant. To avoid this, exclude those device platforms from such policies. This does not apply to a block-legacy policy, but it matters if you test other policies the same way.
Limits
- The thread explains the symptom, but it cannot tell you whether your tenant has legacy traffic. Your logs can.
- The statement that legacy sign-ins appear as Other clients in non-interactive logs comes from the moderator, and Learn's pages I used do not confirm it as a rule.
- Learn's pages show no retention period for the logs. Check your own.
Related
- Conditional Access
- Conditional Access report-only mode
- Legacy authentication
- Microsoft Entra sign-in logs
Sources
- Microsoft Learn: Block legacy authentication with Conditional Access
- Microsoft Learn: Conditional Access report-only mode
- Microsoft Learn: Troubleshoot Conditional Access with sign-in logs
- Microsoft Learn: Sign-in logs in Microsoft Entra ID
- Microsoft Q&A: How to evaluate the result of a Conditional Access policy (March 2026)