Latest News
0

Using volumes in SQL Server 2019 non-root containers

~

I’ve seen a few people online asking how to use docker named volumes with the new SQL Server 2019 RTM images. Microsoft changed the way SQL runs within a container for the RTM versions, SQL no longer runs as the root user.

This is a good thing but does throw up some issues when mounting volumes to create databases on.

Let’s run through what the issue is and how to overcome it.

Run a container from the 2019 GDR1 ubuntu image: –

docker container run -d `
-p 15789:1433 `
--volume sqlserver:/var/opt/sqlserver `
--env MSSQL_SA_PASSWORD=Testing1122 `
--env ACCEPT_EULA=Y `
--env MSSQL_BACKUP_DIR="/var/opt/sqlserver" `
--env MSSQL_DATA_DIR="/var/opt/sqlserver" `
--env MSSQL_LOG_DIR="/var/opt/sqlserver" `
--name testcontainer `
mcr.microsoft.com/mssql/server:2019-GDR1-ubuntu-16.04

What we’re going here is mounting a named volume called sqlserver to /var/opt/sqlserver within the container. Then we’re setting the default data, log, and backup location to /var/opt/sqlserver/.

Now if we try and create a database using those defaults: –

CREATE DATABASE [TestDatabase];
GO

Msg 5123, Level 16, State 1, Line 1
CREATE FILE encountered operating system error 2(The system cannot find the file specified.) while attempting to open or create the physical file ‘/var/opt/sqlserver/testdatabase.mdf’.
Msg 1802, Level 16, State 4, Line 1
CREATE DATABASE failed. Some file names listed could not be created. Check related errors.

We get an error message as the SQL instance within the container does not have access to that location because it’s running as the mssql user.

We need to grant the mssql user access to that folder: –

docker exec -u 0 testcontainer bash -c "chown mssql /var/opt/sqlserver"

This will make the mssql user the owner of that folder. -u 0 sets the command to run as the root user and it has access to be able to change the owner of the folder. For more info on docker exec click here.

So we can now create the database: –

However, we would have to run that command every time we spin up a container with named volumes mounted. A better way would be to create a custom image from a Dockerfile that has created that folder within the container and granted the mssql user access: –

FROM mcr.microsoft.com/mssql/server:2019-GDR1-ubuntu-16.04

USER root

RUN mkdir /var/opt/sqlserver

RUN chown mssql /var/opt/sqlserver

ENV MSSQL_BACKUP_DIR="/var/opt/sqlserver"
ENV MSSQL_DATA_DIR="/var/opt/sqlserver"
ENV MSSQL_LOG_DIR="/var/opt/sqlserver"

USER mssql

CMD /opt/mssql/bin/sqlservr

We’re using the USER command to switch to the root user in order to grant access to the folder and then switching back to the mssql user to run SQL.

Create the custom image: –

docker build -t custom2019image .

Now we can run a container from that image: –

docker container run -d `
-p 15789:1433 `
--volume sqlserver:/var/opt/sqlserver `
--env MSSQL_SA_PASSWORD=Testing1122 `
--env ACCEPT_EULA=Y `
--name testcontainer `
custom2019image

And create the database without having to run anything else: –

CREATE DATABASE [TestDatabase];
GO

Hope that helps!

0

Using the GitHub Package Registry to store container images

~

UPDATE – October 2020 – Github has now released the Github Container Registry which can be used to store container images. For more information, see here

The GitHub Package Registry is available for beta testing and allows us to store container images in it, basically giving us the same functionality as the Docker Hub.

However the Docker Hub only allows for one private repository per free account whereas the Github package registry is completely private! Let’s run through a simple demo to create a registry and upload an image.

First thing to do is create a personal access token in GitHub. Go to Settings > Developer Settings > Personal Access Tokens

Ensure that the token has the rights set above and click Generate Token

Now we can use that token to login to the package registry: –

TOKEN=XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX
docker login docker.pkg.github.com -u dbafromthecold -p $TOKEN

Search for a test image. I’m going to use the busybox image which is 2MB: –

docker search busybox

Then pull the image down: –

docker pull busybox:latest

Tag the image with the repo name to be push to. The format is docker.pkg.github.com/USERNAME/REPOSITORY/IMAGE:TAG

docker tag busybox:latest docker.pkg.github.com/dbafromthecold/testrepository/busybox:latest

N.B. – the repo used has to already exist within your github account

Now push the image to the GitHub Package repository: –

docker push docker.pkg.github.com/dbafromthecold/testrepository/busybox:latest

And then you should be able to see the package in GitHub: –

Thanks for reading!

4

Running SQL Server containers as non-root

~

Recently I noticed that Microsoft uploaded a new dockerfile to the mssql-docker repository on Github. This dockerfile was under the mssql-server-linux-non-root directory and (you guessed it) allows SQL Server containers to run as non-root.

But why is running a container as root bad? Let’s run through an example.

Using a non-root user: –

Run a SQL Server 2019 container with /etc mounted:-

docker run -d -p 15789:1433 \
--volume /etc:/etc \
--env SA_PASSWORD=Testing1122 \
--env ACCEPT_EULA=Y \
--name testcontainer \
mcr.microsoft.com/mssql/server:2019-RC1-ubuntu

Have a look at the logs: –

docker logs testcontainer

So even though I ran the container as a non-root user, the container is running as root.

Here’s the reason that’s bad. Exec into the container: –

docker exec -it testcontainer bash

Now create a user and add to the super user’s group: –

useradd testuser
passwd testuser
adduser testuser sudo

The user has been created and added to the super user’s group within the container. But if we come out of the container and run: –

cat /etc/group | grep sudo

The user is in the super user group on the host! Which means we can do: –

su testuser

Because we mounted the /etc directory into the container, the user created in the container is also created on the host!

And that’s why running containers as root is bad.

EDIT: November 2019

The new SQL Server 2019 run as a non-root user by default, these images are: –

mcr.microsoft.com/mssql/server:2019-GA-ubuntu-16.04
mcr.microsoft.com/mssql/server:2019-GDR1-ubuntu-16.04

So there’s no need to build your own image but the process below will show you how to (if you want to see how it’s done).

Let’s fix this by running SQL Server 2019 in a non-root container. First thing to do is create a mssql user on the host (you’ll have to run this as a user with sudo rights): –

useradd -M -s /bin/bash -u 10001 -g 0 mssql

N.B. – this user is needed as it’s created in the dockerfile, without it on the host the build will complete but any containers created from the image will crash.

Now, build the image from the dockerfile on Github: –

docker build -t 2019-nonroot .

Let’s try to run this container with /etc mounted: –

docker run -d -p 15799:1433 \
--volume /etc:/etc \
--env SA_PASSWORD=Testing1122 \
--env ACCEPT_EULA=Y \
--name testcontainer2 \
2019-nonroot

We can see that the container is running as the user mssql and it’s errored out as it does not have access to the /etc directory that we tried to mount!

So now that we have the option to run SQL Server in containers as a non-root user, I would absolutely recommend that you do so.

Thanks for reading!

7

Use port forwarding to access SQL Server running in Kubernetes

~

A really handy feature in Kubernetes is port forwarding. This can be used to narrow down an issue when connections are failing to SQL Server running in a cluster.

Say we have deployed the following to a Kubernetes cluster: –

apiVersion: apps/v1beta1
kind: Deployment
metadata:
  name: sqlserver
spec:
  replicas: 1
  template:
    metadata:
      labels:
        name: sqlserver
    spec:
      containers:
      - name: sqlserver1
        image: mcr.microsoft.com/mssql/server:2019-RC1-ubuntu
        ports:
        - containerPort: 1433
        env:
        - name: SA_PASSWORD
          value: "Testing1122"
        - name: ACCEPT_EULA
          value: "Y"
---
apiVersion: v1
kind: Service
metadata:
  name: sqlserver-service
spec:
  ports:
  - name: sqlserver
    port: 1433
    targetPort: 1433
  selector:
    name: sqlserver
  type: LoadBalancer

This will create the following in the Kubernetes cluster: –

The load balanced service’s IP can be usually be used to connect into the SQL instance running in the pod, but what if we’re unable to connect? Does the issue lie with the service or the pod?

In order to narrow this down, port forwarding can be used to directly connect to the pod: –

kubectl port-forward pod/sqlserver-889b56d7b-nb2b4 15789:1433

This will allow us to use 127.0.0.1,15789 (localhost won’t work) and connect from our local machine to the pod running in the Kubernetes cluster (in a separate window): –

mssql-cli -S 127.0.0.1,15789 -U sa

We can use the same port to connect via ADS and SSMS as well: –

If a connection can be established to the pod via the forwarded port then we know that the issue doesn’t lie with the pod but with the service or the connection from the service to the pod.

Thanks for reading!

0

Data Relay 2019

~

My favourite UK Data Platform event is back for 2019 and has just opened for registration!

If you haven’t heard of Data Relay before, it’s pretty unique. It’s a 5 day conference but each of those 5 days is in a different city. Typically cities that don’t host their own data event.

This means that after each day, the event packs up and moves on. As you can imagine this poses quite a few challenges but it’s extremely satisfying to see it all come together.

The reason I like Data Relay so much is that it really fosters a sense of camaraderie between everyone involved. We’re all in it together to get each event up and running every day.

That’s how I got involved in the first place. I got selected to speak at each event in 2017 and ended up helping out with packing swag bags, moving boxes, and generally doing anything else that needed to be done. Have to admit, it was exhausting but really really good fun.

So if you’d like to come to one of the best community events on the calendar, this year’s schedule is: –

  • Newcastle – October 7th – Register here
  • Leeds – October 8th – Register here
  • Nottingham – October 9th – Register here
  • Birmingham – October 10th – Register here
  • Bristol – October 11th – Register here

Thanks for reading and hope to see you at Data Relay 2019!