Entra ID Private Access with private integrated storage accounts
Introduction
In the past couple of weeks, I worked on a project where I needed to provide access to a securely private integrated Azure Storage Account via the Entra ID Private access profile. During this project I encountered a very interesting bug, that made me better understand the insights of how Global Secure Access works.
The setup
The requirement for this setup is that access to the Storage Account was provided by using the Azure Storage Explorer application. Via this applications we needed to tunnel the data through Entra ID Private Access, and allowed it to access the private integrated Storage Account.
While it might seem irrelevant, the endpoints for this scenario were all Entra ID Joined, and there was no Active Directory domain where the Private Access Connector nor the Storage Account could be integrated with. This 'cloud-only' setup is an important factor when we start explaining the workarounds for the bug I encountered.
Additionally, we also needed secure access to a Microsoft Cloud PC via the 'require complaint network' control in conditional access.
Azure networking
To zoom in a little bit on the Azure networking part, we had a hup-spoke network architecture protected by an Azure Firewall in the Hub. The subnet of the private integrated Storage Account was one of the specific spokes, where we needed to provide access to. Below you can find the Private Endpoint configuration for the Storage Account:
Since the Azure Storage Explorer is used to connect to the Storage Account, we knew that we needed the FQDN of the Private Integrated Storage Account to send the tunneled traffic to. This FQDN can be found in the DNS configuration of the Private Endpoint:
Private Access profile
To configure the Private Access profile, we choose to deploy a new Global Secure Access application specifically for this scenario. After installing the Private Access connector in Azure and choosing the correct connector group, we configured the FQDN of the Private Endpoint into the Private Access profile:
After assigning the correct users to both the application and the Private Access profile, we were able to connect to the Storage Account via the Azure Storage Explorer.
The bug
When we enabled the Microsoft Entra Private Access profile, the connection to the private integrated Storage Account worked. But once we enabled the Microsoft profile as well, the connection stopped working.
Troubleshooting
After troubleshooting this issue we found that Microsoft pushed extra tunneling rules in the background, which you do not see in the GUI of the client or in the Entra ID portal. One of these rules was a rule setting a bypass action to `blob.core.windows.net` traffic for the Microsoft profile:
Even though the rule was set to bypass, the rule we created for the Private Access profile was ignored since the rule for the Microsoft profile had a higher priority. (order for Microsoft profile rule is 54, while order for Private Access rule is 91).
This was a very interesting finding for me, as Microsoft pushes more rules in the background than we can see in Entra ID or the GSA Client. It was also interesting to see how a rule which was set to bypass, still broke other related rules because of the priority (which was logical afterwards). On the day of writing this blog, there is no way to change the orders of these rules ourselves.
Tested workarounds
To try and solve this issue, we tested multiple workarounds.
Custom domain on Azure Storage Account
The first workaround we investigated is whether we could setup a custom domain on the Azure Storage Account, so that we can tunnel that specific custom domain rather than the blob.core.windows.net
FQDN.
The problem with this workaround is that we would need to allow HTTP access to the Storage Account, if we do not want to use and Azure Front Door. This is because custom domains on Azure Storage Accounts with HTTPS enabled are only supported when using an Azure Front Door.
For our scenario, disabling HTTPS was not an option, and deploying an Azure Front Door would make the setup more complex than needed. But it might be worth a shot if you are not facing these technical requirements 😉.
Private DNS (preview)
In Global Secure Access there is a new feature (currently in preview) which is called Private DNS. With this feature you can query your own internal DNS servers to resolve IP addresses for internal domain names.
We tried using this feature where we entered the DNS suffix of the Azure private DNS zone into the Private DNS profile. However, this also did not seem to work.
Hosts file
One of the easiest but less scalable workaround was to change the hosts file on the devices, and map the FQDN of the Private Endpoint to the IP address it relates to.
******ftransfer001.blob.core.windows.net 10.40.15.4
c:\Windows\System32\Drivers\etc\hosts
For this to work, the IP address needs to be tunneled in the Private Access profile instead of the FQDN:
Once this is done, the hosts file takes precedence in resolving the IP address related to the FQDN, and tunnels the traffic based on IP address rather than FQDN which bypasses the Microsoft Access Profile. A smart solution for a limited amount of devices, but not scalable for a large setup.
The fix
For the issue to be eventually fixed, Microsoft removed the blob.core.windows.net
rule from the Microsoft Profile in the backend for our Microsoft tenant (as discussed in the troubleshooting section). This is something which can only be done via Microsoft Support.
If you are facing the same issue, I would recommend creating a support request with Microsoft. They did let me know that they are planning to remove this rule in the Microsoft Profile for all tenants, but there was no ETA they could give for that.
Conclusion
I wanted to share this use-case with the public, as it shows how Entra ID Private Access can be used in a lot of specific scenario's just like the scenario we discussed. In the past we had to provide access to private integrated applications in Azure via an Azure VPN to end-users, which introduced latency issues. These issues seem to have gone away by using Global Secure Access in our case!
This blogpost also shows how Microsoft is doing a tremendous job at maturing this relatively new product, and is pushing it to more to a production ready solution.