Report-only mode for conditional access

This week is, like last week, about a awareness for new feature that is introduced with conditional access. Last week was all about the recently introduced Conditional Access Insights workbook. In that post I already mentioned the Report-only mode for conditional access policies. In this post I want to focus on that Report-only mode. Report-only mode is a new state of a conditional access policy state that allows IT administrators to evaluate the impact of conditional access policies before enabling them in their environment. That enables the IT administrators to anticipate on the number and names of users impacted by common deployment initiatives such as blocking legacy authentication, requiring multi-factor authentication, or implementing sign-in risk policies. A great step forward.

In this post I’ll walk through the steps of configuring Report-only mode for conditional access policies, followed by looking at the end-user experience. I’ll end this post by looking at the administrator experience.

Configure report-only mode

Let’s start by having a look at the steps to configure the Report-only mode for a conditional access policy. These steps will walk through the creation of a new conditional access policy, with a focus on configuring the Report-only mode. The exact configuration of the conditional access policy assignments and conditions are not part of that focus. The following three steps walk through that configuration.

  1. Open the Azure portal and navigate to Azure Active Directory  > Security > Conditional access (or open the Microsoft 365 Device Management portal and navigate to Endpoint security Conditional access) to open the Conditional access – Policies blade
  2. On the Conditional access – Policies blade, click New policy to open the New blade
  3. On the New blade, configure the assignment and conditions to filter the users and cloud apps that should be targeted by the conditional access policy. After configuring the conditions it’s time to look at the access controls. The access controls are the configuration that eventually might impact the end-user. In the access controls, the grant control determines that behavior. In the grant control the IT administrator can configure the requirements that should be met for accessing the cloud app for the end-user. Depending on the configured requirements, there might be a minimal impact for the end-user (see Figure 1 and and Figure 2 about the messages that are shown about the impact of the conditional access policy based on the configured requirements). After configuring the grant control, select Report-only with Enable policy (also shown in Figure 1) and click Create.

End-user experience

Depending on the configuration that is used in the grant control, of the conditional access policy, the end-user might have a slight impact when using the Report-only mode. The table below is a summary of the available requirements in combination with the potential impact. This table is based on the information as shown during the configuration of the conditional access (see Figure 2), as I haven’t been able to get the mentioned experience on my test devices. I’ve tested with a Samsung Galaxy 10, iPad 2018 and iPhone X.

RequirementPotential user impact
Require multi-factor authentication No impact
Require device to be marked as compliantMay prompt users on macOS, iOS and Android devices to select a device certificate
Require Hybrid Azure AD joined device No impact
Require approved client appMay prompt users on macOS, iOS and Android devices to select a device certificate
Require app protection policyMay prompt users on macOS, iOS and Android devices to select a device certificate

Administrator experience

An interesting part to look at is the experience of the IT administrator. That can be achieved by looking at the Conditional Access Insights workbook (as shown last week). The Conditional Access Insights workbook can be used to get the insights of the different Report-only mode conditional access policies. The data in the workbook can be filtered to only show information about Report-only mode conditional access policies, or even only data of a specific conditional access policy.

Besides that workbook, the Sign-ins monitoring of Azure AD also provides a new tab in the details of a sign-in. That tab is the Report-only (Preview) tab. As shown below that tab provides information about the different Report-only mode conditional access policies that were applicable to the sign-in. Per conditional access policy, the result is shown of the sign-in. That result will show what the effect would be of that conditional access policy and that information will help with determining the impact of enabling that conditional access policy.

Below is an overview of the different result states of a Report-only conditional access policy. Almost all of these results are shown in Figure 3 above (with the exception of the user action required result).

ResultExplanation
Report only: FailureThe configured conditional access policy conditions were satisfied, but not all the required (non-interactive) controls were satisfied.
Report only: SuccessThe configured conditional access policy conditions and required (non-interactive) controls were satisfied. 
Report only: Not appliedNot all configured conditional access policy conditions were satisfied.
Report only: User action requiredThe configured conditional policy conditions were satisfied, but a user action would be required to satisfy the required controls.

More information

For more information regarding report-only, please refer to the following documents:

Conditional Access Insights

This week is all about creating awareness for the Conditional Access Insights workbook. This workbook is currently still in preview and is using Azure Monitor workbook functionality. The Conditional Access Insights workbook contains sign-in log queries that can help IT administrators with getting insights on the impact of conditional access policies. That is useful for troubleshooting, for following trends and for testing the latest introduction to conditional access of Report-only policies. Especially the latest category can be easily verified by using the Conditional Access Insights workbook. In this post I’ll walk trough the steps of creating a Log Analytics workspace (to store Azure Monitor log data), followed by the steps to send Azure AD sign-in information to Azure Monitor logs.I’ll end this post by actually looking at the Conditional Access Insights workbook.

Create a Log Analytics workspace

The first step to prepare for using the Conditional Access Insights workbook, is to create a Log Analytics workspace. A Log Analytics workspace is a unique environment for Azure Monitor log data. Each Log Analytics workspace has its own data repository and configuration, and data sources and solutions are configured to store their data in a particular workspace. To create a Log Analytics workspace simply follow the 2 steps below.

  1. Open the Azure portal and navigate to  All services  > Log Analytics workspaces to open the Log Analytics workspaces blade
  2. On the Log Analytics workspaces blade, provide the following information and click OK
  • Select Create New
  • Log Analytics Workspace: Provide a unique name for the Log Analytics workspace
  • Subscription: Select a valid subscription for the Log Analytics workspace
  • Resources group: Select an existing resource group for the Log Analytics workspace, or click Create new to create a new resource group for the Log Analytics workspace
  • Location: Select a location for the Log Analytics workspace
  • Pricing tier: Select a pricing tier for the Log Analytics workspace

Note: Alternatively the Log Analytics workspace can be created during the process of configuring the diagnostic settings of Azure AD.

Send logs to Azure Monitor logs

The second step to prepare for using the Conditional Access Insights workbook, is to send the Azure AD sign-in logs to Azure Monitor logs (previously known as Log Analytics). Azure Monitor logs allows the administrator to query data to find particular events, analyze trends, and perform correlation across various data sources. To send the Azure AD sign-in logs to Azure Monitor logs simply follow the 3 steps below.  

  1. Open the Azure portal and navigate to  Azure Active Directory  > Diagnostic settings to open the [Azure AD] > Diagnostic settings blade
  2. On the [Azure AD] > Diagnostic settings blade, click Add diagnostic settings to open the Diagnostic settings blade
  3. On the Diagnostic settings blade, provide the following information and click Save
  • Name: Provide a unique name for the diagnostic settings configuration
  • Select Send to Log Analytics
  • Subscription: Select a valid subscription for the Azure Monitor logs
  • Log Analytics Workspace: Select the previously created Log Analytics workspace as a location to store the Azure Monitor logs
  • Log: Select SignInLog

Conditional Access Insights workbook

After making sure that the Azure AD sign-in information is send to Azure Monitor logs, the Conditional Access Insights workbook can be used to get insights in the log data. This workbook contains sign-in log queries that can help IT administrators monitor the impact of conditional access policies. This provides the IT administrators with the ability to report on how many users would have been granted or denied access. This workbook contains details per condition so that the impact of a policy can be contextualized per condition. The following steps walk through navigating to and through the Conditional Access Insights workbook.

  1. Open the Azure portal and navigate to  Azure Active Directory  > Workbooks to open the [Azure AD] > Workbooks blade

Tip: Also make sure to take a look at the other available workbooks, as those workbooks provide a lot of insights about the different sign-ins. Really useful for insights.

  1. On the [Azure AD] > Workbooks blade, click Conditional Access Insights (Preview) to open the Conditional Access Insights (Preview) workbook

The Conditional Access Insights workbook provides the IT administrator with a lot of insights based on the Azure AD sign-in information. The figures above show the following information:

  • Figure 4 shows the parameter selection and the Impact summary section of the workbook. The parameter selection section provides five parameters to filter the insights of the workbook: Conditional Access Policy, Time Range, User, Apps and Data View. The first filter can also be used to easily verify the impact of the recently Report-only conditional access policies, as the insights can be filtered to a specific conditional access policy. The Impact summary section, shows a quick overview of the results for the selected conditional access policy in the specified time range. Clicking on the different tiles will further filter the breakdown sections.
  • Figure 5 and 6 show the Breakdown per condition and sign-in status section of the workbook. The Breakdown per condition and sign-in status section shows the impact of the selected conditional access policies broken down by each of six conditions: Device state, Device platform, Client apps, Sign-in risk, Location and Applications. Clicking on the logs sign with a breakdown, will open the used query in the logs viewer. That will provide the kql-query that is used to filter the right information.
  • Figure 7 shows the Sign-in details section of the workbook. The Sign-in details section enables the IT administrator to investigate individual sign-ins, filtered by the parameters selected in the workbook. Search for individual users, sorted by sign-in frequency, and view their corresponding sign-in events.

More information

For more information regarding conditional access insights, refer to the following documents:

Working with the restart behavior of Win32 apps

A long time ago, I did a post about Working with the restart behavior of Applications in ConfigMgr 2012. That post is still being read pretty well. Based on the interest of that post, and the introduction of nice new features to the Win32 apps, I thought it would be a good idea to redo that post for Microsoft Intune. Before an IT administrator had to be creative to work with, or work around, the restart behavior of Win32 apps. Either by wrapping installations and capturing the exit code, or by tuning the translation of an return code. With the latest adjustments to the Win32 apps, within Microsoft Intune, the IT administrator has more options to actually work with the return code of an Win32 app installation. These configuration options are similar to the configuration options within the app model of ConfigMgr. In this post I’ll discuss the 2 layers that together define the restart behavior after the installation of Win32 apps.

Return codes

When looking at the restart behavior after the installation of Win32 apps, the first thing that should be looked at is the return code after the installation. By default, when adding a Win32 app to Microsoft Intune, a list of standard return codes is added to indicate post-installation behavior (see figure below). These are often used return codes. When the Win32 app installation ends with a different return code, additional entries can be added. This configuration is available via [Win32 app] > Properties > Return codes.

Fore every return code a code type can be configured. The code configures the post-installation behavior of the Win32 app. The following code types are available and can be configured with the return code to apply the mentioned behavior:

  • Failed – The Failed return code indicates that the Win32 app installation failed.
  • Hard reboot – The Hard reboot return code indicates that the device is required to restart to complete the installation. Additional Win32 apps cannot be installed on the device without restart. The user will be notified about the required restart.
  • Soft reboot – The Soft reboot return code indicates that the next Win32 app is allowed to be installed without requiring a restart, but a restart is necessary to complete the installation of the installed Win32 app. The user will be notified about the restart.
  • Retry – The Retry return code indicates that the Win32 app installation is retried three times. The installation will wait for 5 minutes between each attempt.
  • Success – The Success return code indicates the Win32 app installation was successful.

Enforce device restart behavior

The second thing that should be looked at is how the device will react on the configured return code. By default, when adding a Win32 app to Microsoft Intune, the default device restart behavior is set to App install may force a device restart (see figure below). This configuration will make sure that the device will restart after the Win32 app installation, if needed, but still in an acceptable manner. The restart behavior can be configured to respond to return code differently. That configuration is available via [Win32 app] > Properties > Program.

Multiple device restart behavior configurations are available. And all these configuration options have their own effect on the return code of the Win32 app installation. The following device restart behaviors are available and can be configured to apply the mentioned behavior (including a short explanation about the expected behavior):

  • Determine behavior based on return codes: This option means that the device will restart based on the configured return code. With this configuration a Hard reboot return code will immediately trigger a restart of the device and a Soft reboot return code will notify the user that a restart is required to finish the installation.
  • No specific action: This option means that the installation will suppress device restarts during the Win32 app installation of MSI-based apps. Effectively that means that parameters are added to the installation command line of MSI-based apps to suppress device restart. With this configuration a Hard reboot return code will notify the user that a restart of the device will be triggered in 120 minutes and a Soft reboot return code will notify the user that a restart is required to finish the installation.
  • App install may force a device restart: This option means that the Win32 app installation is allowed to complete without suppressing restarts. With this configuration a Hard reboot return code will notify the user that a restart of the device will be triggered in 120 minutes and a Soft reboot return code will notify the user that a restart is required to finish the installation.
  • Intune will force a mandatory device restart: This option means that a successful Win32 app installation will always restart the device. With this configuration any successful return code will immediately trigger a restart of the device.

More information

For more information about the Win32 app functionality in Microsoft Intune, refer to the documentation about Intune Standalone – Win32 app management.

Applicability rules for device configuration profiles

This week a new blog post about a little nice, but quite unknown, feature. Applicability rules for device configuration profiles. The nice thing about applicability rules is that those rules can be used to target devices in a group that meet specific criteria. That enables an administrator to assign a device configuration profile to all users, or all Windows 10 devices, but only actually apply to Windows 10 devices of a specific version or edition. In this post I’ll go through the configuration of applicability rules (including a few important details) and the administrator experience.

Configure applicability rule

Let’s start by looking at applicability rules. Applicability rules can be configured for every device configuration profile type with Windows 10 and later as Platform, with the exception of Administrative Templates as Profile Type. It enables the administrator to only assign the device configuration profile to a specific version or edition of Windows 10.

Before looking at the configuration of applicability rules, it’s good to be familiar with a few important notes about assigning a device configuration profile including applicability rules. When assigning such a device configuration profile, keep the following in mind:

  • When two device configuration profiles are assigned with the exact same settings, and only one of those profiles has an applicability rule configured, then the profile without an applicability rule is applied.
  • When assigning device configuration profiles to groups, the applicability rules act as a filter, and only target the devices that meet the specified criteria.

Now let’s have a look at the actual configuration of applicability rules. The following steps walk through the configuration of applicability rules in device configuration profiles for Windows 10 devices.

  1. Open the Microsoft 365 Device Management portal and navigate to Devices > Windows Configuration profiles to open the Windows – Configuration profiles blade
  2. Select an existing device configuration profile, or create a new device configuration profile and navigate to Applicability Rules to open the Applicability Rules blade
  3. On the Applicability Rules blade, configure a rule click Add to add the rule and click Save
  • The Rule selection enables the administrator to either use Assign profile if – that will include users or groups that meet the specified criteria – or use Don’t assign profile if – that will exclude users or groups that meet the specified criteria –.
  • The Property selection enables the administrator to either use OS edition – that will enable a list to check the Windows 10 editions that must be included – or use OS version – that will enable fields to enter the min and max Windows 10 version numbers that must be included –. Both values (min and max) are required.

Administrator experience

Let’s end this post by shortly mentioning the administrator experience. The experience is not that exiting actually. When an applicability rule is applicable to a device, the device is targeted with the configuration profile. The device will try to assign the configuration profile and simply show the normal Succeeded, Error or Failed status. When an applicability rule is not applicable to a device, the device wil not be targeted with the configuration profile and the configuration profile will get the status of Not applicable.

More information

For more information regarding applicability rules for device configuration profiles, refer to the Applicability rules section of the Create a device profile in Microsoft Intune doc.