Azure App Service is a PaaS service used to host a variety of apps on Azure: web apps, API apps, functions, and mobile apps. App Service Environment or ASE allows enterprises to deploy their App Service apps in a subnet in their Azure Virtual Network, providing an isolated, highly scalable, and dedicated environment for their cloud workloads.
This reference architecture demonstrates a common enterprise workload using ASE and best practices to tighten the security of this workload.
Architecture
The main component of this architecture is the App Service environment. An ASE can be deployed either as an external ASE with a public IP address or as an internal ASE with an internal IP address belonging to the Internal Load Balancer (ILB). This reference architecture deploys an enterprise web application in an internal ASE called an ILB ASE. Use an ILB ASE when your scenario requires you to :
Host intranet applications securely in the cloud, which you access through a site-to-site or ExpressRoute.
Protect apps with a WAF device.
Host apps in the cloud that aren't listed in public DNS servers.
Create internet-isolated backend apps which your front-end apps can securely integrate with.
An ASE must always be deployed in its own subnet within the enterprise's virtual network, allowing tight control of the incoming and outgoing traffic. Within this subnet, App Service applications are deployed in one or more App Service plans, a collection of physical resources required to run the app. Depending on the complexity and resource requirement, an App Service plan can be shared between multiple apps.
The following services are key to locking down the ASE in this architecture :
Azure Virtual Network or VNet is a private Azure cloud network owned by the enterprise. It provides network-based security and isolation. ASE is an App Service deployment into a subnet of the enterprise-owned VNet. It allows the enterprise to control that network space and the resources it accesses tightly using Network Security groups and Service Endpoints.
Application Gateway is an application-level web traffic load balancer with TLS/SSL offloading and web application firewall (WAF) protection. It listens on a public IP address and routes traffic to the application endpoint in the ILB ASE. Since this is application-level routing, it can route traffic to multiple apps within the same ILB ASE. See Application Gateway multiple site hosting to learn how App Gateway supports various sites.
Azure Firewall is used to restrict the outbound traffic from the web application. Outgoing traffic that does not go through the service endpoint channels and a route table required by ASE is directed to the firewall subnet. Although it is recommended that service endpoints be configured on the firewall subnet for traceability, it may not always be feasible. For example, some service endpoints are required by the ASE infrastructure to be on the ASE subnet. For simplicity, this architecture configures all service endpoints on the ASE subnet.
Azure Active Directory or Azure AD provides access rights and permissions management to Azure resources and services. Managed Identities assigns identities to services and apps, which are automatically managed by Azure AD. These identities can be used to authenticate to any service that supports Azure AD authentication. This removes the need to explicitly configure credentials for these apps. This architecture assigns a managed identity to the web app.
Key Vault stores any secrets and credentials required by the apps. Use this option over storing secrets directly in the application.
Azure Pipelines provides Continuous Integration and Continuous Deployment capabilities in this architecture. Since the ASE is internal to the virtual network, a virtual machine is used as a jumpbox inside the VNet to deploy apps in the App Service plans. The pipeline builds the apps outside the VNet. For enhanced security and seamless RDP/SSH connectivity, consider using the recently released Azure Bastion as the jumpbox.
Multi-site configuration
Internal ASE can host several web apps and APIs with HTTP endpoints. These applications are locked down from the public internet as the ILB IP is only accessible from within the Virtual Network. The Application Gateway is used to selectively expose these applications to the internet. ASE assigns a default URL to each App Service application as <default-appName>.<ASE-domain>.appserviceenvironment.net. A private DNS zone is created that maps the ASE domain name to the ASE ILB IP address. This avoids using a custom DNS to access the apps within the VNet.
The App Gateway is configured such that a listener listens on the HTTPS port for requests to the Gateway's IP address. For simplicity, this implementation does not use a public DNS name registration and requires you to modify the localhost file on your computer to point an arbitrarily chosen URL to the App Gateway's IP. For simplicity, the listener uses a self-signed certificate to process these requests. The App Gateway has backend pools for each App Service application's default URL. A routing rule is configured to connect the listener to the backend pool. HTTP settings are created to determine whether the connection between the gateway and the ASE will be encrypted. These settings are also used to override the incoming HTTP host header with a hostname picked from the backend pool. This implementation uses default certificates created for the default ASE app URLs, which are trusted by the gateway. The request is redirected to the default URL of the corresponding app. The private DNS linked to the VNet forwards this request to the ILB IP. The ASE then forward this to the requested App service. Any HTTP communication within the ASE apps goes through the private DNS. Note that the listener, backend pool, routing rule, and HTTP settings need to be configured on the App Gateway for each ASE app.
Security considerations
App Service Environment
An internal ASE is deployed in the enterprise virtual network, hidden from the public internet. It allows the enterprise to lock down its backend services, such as web APIs and functions, at the network level. Any ASE app with an HTTP endpoint can be accessed through the ILB from within the virtual network. The App Gateway is configured to forward requests to the web app through the ILB. The web app itself goes through the ILB to access the API. The critical backend components, that is, the API and the function, are effectively inaccessible from the public internet.
Default certificates are created for each app service using the default domain name assigned by ASE. This certificate can tighten the security of the traffic between the gateway and the app and may be required in certain scenarios. These certificates will not be visible by the client browser; they can only respond to the certificates configured at the App Gateway.
App Gateway
Encryption
App Gateway can use Transport Layer Security (TLS)/Secure Sockets Layer (SSL) to encrypt and protect all traffic from web browsers. Encryption can be configured in the following ways:
Encryption terminated at the gateway. The backend pools, in this case, are configured for HTTP. The encryption stops at the gateway, and traffic between the gateway and the app service is unencrypted. Since encryption is CPU-intensive, unencrypted traffic at the backend improves performance and allows simpler certificate management. This provides a reasonable level of security since the backend is secured by virtue of the network configuration.
End-to-end encryption. In some scenarios, such as specific security or compliance requirements, the traffic might be required to be encrypted between the gateway and the app. This is achieved by using an HTTPS connection and configuring certificates at the backend pool.
Web Application Firewall
Web Application Firewall (WAF) on the Application Gateway protects web apps from malicious attacks, such as SQL injection. It is also integrated with Azure Monitor to monitor attacks using a real-time log. WAF needs to be enabled on the gateway.
Virtual Network
Network security groups can be associated with one or more subnets in the VNet. These are security rules that allow or deny traffic to flow between various Azure resources. This architecture associates a separate NSG for each subnet. This allows fine-tuning of these rules per subnet, as per the service(s) contained in that subnet. For example, explore the configuration for "type": "Microsoft. Network/networkSecurityGroups" in the file ase.json for the NSG for the ASE subnet and in the file appgw.json for the NSG for App Gateway subnet.
Firewall
Azure Firewall and service endpoints complement each other. While service endpoints protect the resources by restricting incoming traffic from originating from your virtual network, Azure Firewall allows you to refuse the outbound traffic from your applications. It is recommended to let all outbound traffic pass through the firewall subnet, including service endpoint traffic. This allows better monitoring of the outbound traffic. However, ASE infrastructure requires service endpoints for SQL Server, Storage, and Event Hubs to be configured on the ASE subnet
ASE Management IPs
The ASE management traffic flows through the enterprise virtual network. This traffic should be routed directly to the dedicated IP addresses outside the virtual network, avoiding the firewall. This is achieved by creating a user-defined virtual network route table.
Azure Active Directory
Azure Active Directory provides security features to authenticate applications and authorize access to resources. This reference architecture uses two key features of Azure Active Directory: managed identities and Azure role-based access control.
Azure role-based access control (Azure RBAC) manages access to Azure resources. This includes:
Which entity has the access: user, managed identity, security principal.
What type of access it has: owner, contributor, reader, admin.
The scope of access is defined as a resource, resource group, subscription, or management group.
Key Vault
Some services support managed identities. However, they use Azure RBAC to set up permissions for the app. For example, see Service Bus' built-in roles and Azure RBAC in Azure Cosmos DB. Owner access to the subscription is required to grant these permissions, even though personnel with contributor access can deploy these services.
Scalability considerations
Design scalable apps
The application in this reference architecture is structured so individual components can be scaled based on usage. Each web app, API, and function is deployed in its App Service plan. You can monitor each app for performance bottlenecks and then scale it up if required.
Autoscaling App Gateway
Autoscaling can be enabled on Azure Application Gateway V2. This allows App Gateway to scale up or down based on the traffic load patterns. This reference architecture configures autoscaleConfiguration in the file appgw.json to scale between zero and ten additional instances.
Deployment considerations
Apps can be deployed to an internal ASE only from within the virtual network. The following three methods are widely used to deploy ASE apps:
1. Manually inside the Virtual Network:
Create a virtual machine inside the ASE Virtual Network with the required tools for the deployment. Open up the RDP connection to the VM using an NSG configuration. Copy your code artefacts to the VM, build, and deploy to the ASE subnet. This is a simple way to set up an initial build and test development environment. It is, however, not recommended for the production environment since it cannot scale the required deployment throughput.
2. Point to-site connection from the local workstation:
This allows you to extend your ASE VNet to your development machine and deploy from there. This is another way to set up an initial dev environment, which is not recommended for production.
3. Through Azure Pipelines:
This implements a complete CI/CD pipeline, ending in an agent inside the VNet. This is ideal for production environments requiring high throughput of deployment. The build pipeline remains entirely outside the VNet. The deploy pipeline copies the built objects to the build agent inside the VNet and then deploys to the ASE subnet.
Cost considerations
Use the Azure pricing calculator to estimate costs. Other considerations are described in the Cost section in Microsoft Azure Well-Architected Framework. Azure Reservations help you save money by committing to one-year or three-year plans for many Azure resources.
Here are some points to consider for critical services used in this architecture.
App Service Environment
There are various pricing options available for App Service. An App Service environment is deployed using the Isolated Service Plan. This plan has three options for CPU sizes - I1, I2, and I3. This reference implementation uses three I1s per instance.
Application Gateway
Application Gateway pricing provides various pricing options for App Gateway. We use the Application Gateway Standard_v2 and WAF_v2 SKU, which supports auto-scaling and zone redundancy.
Azure Cache for Redis
Azure Cache for Redis pricing provides various pricing options for this service. This architecture uses the Premium SKU for virtual network support.
The following are pricing pages for other services used to lock down the ASE:
Comments