IT Nerd Space

Azure App Service Architecture (3): App Service on Linux

Azure App Service Architecture (3): App Service on Linux

Azure App Service Architecture (3): App Service on Linux

Until now, Azure customers could deploy their Web Applications running PHP, Node.js,… on Windows server running IIS, but now they will have a choice to run them on Linux: Microsoft recently announced the availability, in Public Preview mode, of App Service on Linux:

App Service on Linux is currently in Public Preview and enables customers to run their web apps natively on a Linux platform. This allows for better application compatibility for certain kinds of applications and makes it easier to migrate existing web apps hosted on a Linux platform elsewhere onto Azure App Services.

In this third post in my series on Azure App Service Architecture, I’ll focus on how Microsoft implemented this new Platform as a Service product. What will actually run your app if you deploy it on Linux, what Linux distribution they have chose, and how are all the apps deployed on those underlying Linux servers. If you haven’t read my preview two articles, I strongly recommend you read them first: Azure App Service Architecture (1) and Azure App Service Architecture (2) as they will explain some basic concepts that are used in this one.

Provisioning a Web App on Linux

This part is really trivial. I’ll show some screenshots, but they are self explaining. Let’s click on the [+] button, and head to [Web + Mobile]. From there, we’ll select Web App on Linux (Preview):

2016-11-02-13_51_43-choose-your-pricing-tier-microsoft-azure

I’ll set a new unique name for my new Web App:

2016-11-02-13_51_51-choose-your-pricing-tier-microsoft-azure

To be able to explore a little further the options available to us, I’ll choose to create a new App Service Plan, instead of using the default recommended to me. Notice that while the product is in Preview, it is not available in all Azure regions yet. In Europe it’s only available in West Europe for example. That’s enough for me to test it anyway.

2016-11-02-13_52_13-choose-your-pricing-tier-microsoft-azure

Particularly, I don’t need to pay for a Standard plan, so I’ll choose a Basic B1 here.

2016-11-02-13_52_31-web-app-on-linux-preview-microsoft-azure

So here we go, just click the Create button, and in less than a minute, you’ll have a Web App up and running, ready for you to use to deploy your app, using your favorite Deployment method, Git, FTP…

App Service on Linux Architecture

To better understand what has been deployed by Azure, let’s head to the Development tools section of the Web App setting menu, and select Advanced Tools:

2016-11-02-14_20_37-advanced-tools-microsoft-azure

Then click on the Go link. It will open the traditional Kudu tools console, where we will have access to some internal information regarding our Web App.

Application layer

First let’s figure out what we have at the Application layer. That is, what kind of software is going to serve our application. In this case, from the Environment tab, we can see some interesting Server Environment Variables:

SERVER_SOFTWARE=Apache/2.4.10 (Debian)

So apparently the Web App is deployed on Apache 2.4 on Debian.

Let’s dig a little deeper with the Bash console, and have a look at the processes we can see:

Kudu Remote Execution Console
Type 'exit' to reset this console.
/home> ps -ef
UID PID PPID C STIME TTY TIME CMD
1001 1 0 0 12:54 ? 00:00:00 /bin/sh -c /usr/sbin/apache2ctl -D FOREGROUND
1001 5 1 0 12:54 ? 00:00:00 /bin/sh /usr/sbin/apache2ctl -D FOREGROUND
1001 7 5 0 12:54 ? 00:00:00 /usr/sbin/apache2 -D FOREGROUND
1001 9 1 0 12:54 ? 00:00:00 /usr/bin/mono /usr/lib/mono/4.5/mod-mono-server4.exe --filename /tmp/.mod_mono_server4 --nonstop --appconfigdir /etc/mono-server4
1001 12 7 0 12:54 ? 00:00:00 /usr/sbin/apache2 -D FOREGROUND
1001 13 7 0 12:54 ? 00:00:00 /usr/sbin/apache2 -D FOREGROUND
1001 71 1 6 12:54 ? 00:00:08 /usr/bin/mono /usr/lib/mono/4.5/mod-mono-server4.exe --filename /tmp/mod_mono_server_default --applications /:/opt/Kudu --nonstop
1001 126 71 0 12:56 ? 00:00:00 /bin/bash -c ps -ef && echo && pwd
1001 127 126 0 12:56 ? 00:00:00 ps -ef

We can see very few processes running, and especially interesting, the PID 1 is apache2ctl, so we are most likely running is a container (if we were in a full server, it would be “init”). All the processes run as a non-root user (uid 1001).

We can indeed confirm that our Web App is running inside a Docker container by looking at /proc/1/cgroup:

/home> cat /proc/1/cgroup
11:memory:/docker/fb05a97d54930566111197c8959a5a33ac9af29b6491bf1ca8158b18df50264b
10:cpuset:/docker/fb05a97d54930566111197c8959a5a33ac9af29b6491bf1ca8158b18df50264b
9:hugetlb:/docker/fb05a97d54930566111197c8959a5a33ac9af29b6491bf1ca8158b18df50264b
8:blkio:/docker/fb05a97d54930566111197c8959a5a33ac9af29b6491bf1ca8158b18df50264b
7:perf_event:/docker/fb05a97d54930566111197c8959a5a33ac9af29b6491bf1ca8158b18df50264b
6:net_cls,net_prio:/docker/fb05a97d54930566111197c8959a5a33ac9af29b6491bf1ca8158b18df50264b
5:pids:/docker/fb05a97d54930566111197c8959a5a33ac9af29b6491bf1ca8158b18df50264b
4:cpu,cpuacct:/docker/fb05a97d54930566111197c8959a5a33ac9af29b6491bf1ca8158b18df50264b
3:freezer:/docker/fb05a97d54930566111197c8959a5a33ac9af29b6491bf1ca8158b18df50264b
2:devices:/docker/fb05a97d54930566111197c8959a5a33ac9af29b6491bf1ca8158b18df50264b
1:name=systemd:/docker/fb05a97d54930566111197c8959a5a33ac9af29b6491bf1ca8158b18df50264b

So, Web App on Linux are deployed in Docker containers (and we know the ID of the container in the host).

Unfortunately, there’s not much we can do from inside the Docker container itself to guess anything about the host running the Docker engine, so we don’t really know what is the OS flavor, version or anything else.

We can have a look at the kernel, which is shared between the VM (host) and all the containers run in the same VM:

/home> uname -a
Linux e30a13645e09 4.4.0-45-generic #66-Ubuntu SMP Wed Oct 19 14:12:37 UTC 2016 x86_64 GNU/Linux
So it looks like it it might be some version of Ubuntu, likely 16.04 (Xenial), where 4.4.0-45-generic #66 is available (although it could also be 14.04). Confirmed by Nalim, Software Engineer at Microsoft, they are using Ubuntu 16.04 for the hosts (VM instances), see comments below.

Regarding the Docker image that was used, we can see it’s based on Debian 8 (Jessie):

/home> cat /etc/os-release
PRETTY_NAME="Debian GNU/Linux 8 (jessie)"
NAME="Debian GNU/Linux"
VERSION_ID="8"
VERSION="8 (jessie)"
ID=debian
HOME_URL="http://www.debian.org/"
SUPPORT_URL="http://www.debian.org/support"
BUG_REPORT_URL="https://bugs.debian.org/"

Data layer: Persistent Storage?

Storage inside a Docker container is, usually, volatile, but if we deploy our App there, we don’t want to loose it when the container stops or the host reboots. Also, we want to be able to scale out our Web App when load increase. So we require some kind of persistency!

Similarly to Web App on Windows, where the persisted files are found in D:\home, here we’ll store the Persisted files in /home. As this Web App is recently created, it’s actually quite empty right now:

/home>pwd
/home
/home>ls
LogFiles site

So how is the storage actually persisted?

/home> mount | grep /home
//10.0.176.8/volume-46-default/20a35f31f26928286ecf/580efcc549c040228b254d82ab4ed6e1 on /home type cifs (rw,relatime,vers=3.0,sec=ntlmssp,cache=strict,username=dummyadmin,domain=RD0003FF1A594C,uid=1001,forceuid,gid=1002,forcegid,addr=10.0.176.8,file_mode=0700,dir_mode=0700,nounix,serverino,mapposix,mfsymlinks,rsize=1048576,wsize=1048576,actimeo=1)

We can see that the /home directory is actually a network shared volume mounted via CIFS protocol (I can only assume here it’s backed by Azure Storage). That way Azure will be able to scale out our Web App, deploying more containers based on the very same stateless image, mounting the same volume which is where our Web App files are deployed.

I think Microsoft might be using here a custom implementation of their Docker Volume Plugin for Azure File Storage. (I haven’t had time yet to play with it and see how similar (or not) things look with this Docker storage plugin). For the persistent storage, Microsoft is using Fileservers that export SMB shares mounted in the host. Those are then mapped as /home into the corresponding docker containers.

Putting everything together

Let step back now. This is how I see Microsoft implemented App Service on Linux. First when we only deploy 1 web app on a single instance:

Azure App Service Architecture on Linux (single App & Instance)

Now, how does it look likes if we deploy multiple Web Apps in the same plan, and we scale out the Plan to two instances?

Azure App Service Architecture on Linux (multiple Apps & Instances)

I hope this gave you a good understanding of how Microsoft implemented internally this new App Service on Linux product.