Changing default location for docker containers

A question that regularly comes up when I talk about containers is, “can you specify where the containers/images live on the host?”

This is a good question as the install for docker is great because it’s so simple but bad because well, you don’t get many options to configure.

It makes sense that you’d want to move the containers/images off the C:\ drive for many reasons such as leaving the drive for the OS only but also, say you have a super fast SSD on your host that you want to utilise? OK, spinning up containers is quick but that doesn’t mean we can’t make it faster!

So, can you move the location of container and images on the host?

Well, yes!

There’s a switch that you can use when starting up the docker service that will allow you to specify the container/image backend. That switch is -g

Now, I’ve gone the route of not altering the existing service but creating a new one with the -g switch. Mainly because I’m testing and like rollback options but also because I found it easier to do it this way.

So the default location for containers and images is: – C:\ProgramData\docker

OK, let’s run through the commands to create a new service pointing the container/images backend to a custom location.

First we’ll create a new directory on the new drive to host the containers (I’m going to use a location on the E: drive on my host as I’m working in Azure and D: is assigned to the temporary storage drive): –

new-item E:\Containers -type directory

Now stop the existing docker service and disable it: –

stop-service Docker

set-service Docker -StartupType Disabled

get-service Docker

Now we’re going to create a new service pointing the container backend to the new location: –

new-service -name Docker2 -BinaryPathName "C:\Program Files\docker\dockerd.exe -g E:\Containers --run-service" -StartupType Automatic

Now start the new service up: –

start-service Docker2

get-service Docker2

And check the new location: –

Cool, the service has generated the required folder structure upon startup and any new images/containers will be stored here.

Once thing to mention is that if you have images and containers in the old location they won’t be available to the new service. I’ve tried copying the files and folder in C:\ProgramData\docker to the new location but keep getting access denied errors on the windowsfilter folder.

To be honest, I haven’t spent much time on that as if you want to migrate your images from the old service to the new one you can export out and then load in by following the instructions here.

Thanks for reading!

Copying files from/to a container

Last week I was having an issue with a SQL install within a container and to fix I needed to copy the setup log files out of the container onto the host so that I could review.

But how do you copy files out of a container?

Well, thankfully there’s the docker cp command. A really simple command that let’s you copy whatever files you need out of a running container into a specified directory on the host.

I’ll run through a quick demo but I won’t install SQL, I’ll use an existing SQL image and grab its Summary.txt file.

If you don’t have the 2017 SQL image, you can pull it from the docker hub by running: –

docker pull microsoft/mssql-server-windows

Once you have the image, execute the following to spin up a container: –

docker run -d -p 15789:1433 --env ACCEPT_EULA=Y --env sa_password=Testing11@@ --name testcontainer microsoft/mssql-server-windows

Excellent! Now we can open up a powershell session within the container: –

docker exec -it testcontainer powershell

Once we’re in we can verify where the file is: –

cd "C:\Program Files\Microsoft SQL Server\140\Setup Bootstrap\Log\"

ls

Now exit out of the powershell session within the container. What we’re going to do is copy the Summary.txt file from the container into the C:\temp directory on the host. To do this run (on the host): –

docker cp testcontainer:"C:\Program Files\Microsoft SQL Server\140\Setup Bootstrap\Log\Summary.txt" C:\temp

Cool! Now we have the file on the host and can review.

Of course this also works for copying files into a container. Say we want to copy test.txt from C:\temp on our host into C:\ in the container. We simply run: –

docker cp C:\temp\test.txt testcontainer:C:\

Nice and easy! All we need to remember is that we always specify the source directory in the cp command first.

Thanks for reading!

Connecting a powershell session to a container

One of the questions that I was asked at SQL Saturday Iceland was “how can I view the filesystem within a container?”.

This is a great question as one of the things that people find off-putting about containers is their opaqueness. It’s not obvious where everything lives within the container or how we can view the files within it.

Thankfully there’s a simple docker command that allows us to open a powershell session within a container, that command is docker exec.

Let’s run through an example. Firstly I’m going to spin up a new container running an instance of SQL Server: –

docker run -d -p 15777:1433 --env ACCEPT_EULA=Y --env sa_password=Testing11@@ --name testcontainer microsoft/mssql-server-windows

Then verify that the container is running: –

docker ps

Now, to open up the powershell session inside run: –

docker exec -it testcontainer powershell

N.B. – the -i flag keeps the powershell session open, otherwise it will immediately exit. The -t flag creates a pseudo terminal, you don’t have to include it tbh but the examples I’ve seen online always include it.

Ok, we know have a powershell session within the container and we can execute commands as normal, for example: –

get-psdrive

This will allow us to explore what the filesystem is within the container and run powershell commands as needed. In the screenshot above it’s interesting to see the space available/used on the C: drive. Have a dig around within your containers and let me know if you find anything interesting 🙂

Thanks for reading!

Summary of my Container Series


All my blogs posts about running SQL Server in a Docker container have been collated into The SQL Server and Containers Guide.

If you’re working with SQL Server and want to get started with containers, check it out!


Getting Started

Running your first container

Building a custom image

Pushing images to the Docker Hub

Using WinDocks to run earlier versions of SQL Server in Containers

Creating SQL Containers from a Dockerfile

Running Linux Containers on Windows 10

SQL Server in Kubernetes on Docker for Windows


SQL Server 2019

Running SQL Server 2019 CTP in a Docker container

Displaying the tags within the SQL Server docker repository

Running SQL Server containers as non-root

Using volumes in SQL Server 2019 non-root containers


Webinars

GroupBy Conference – SQL Server & Containers

24HOP – Manage SQL Server in Linux Containers using dbatools


Docker Commands

Viewing container logs

Sharing Container Images

Copying files from/to a container

Parsing Docker Commands

Connecting a powershell session to a container

Automatically restarting Docker Containers

The Docker kill command

Docker Debug


Configuration & Administration

Attaching databases via a Dockerfile

Attaching databases via a Dockerfile – UPDATE

SQL Containers and Networking

Portainer – A GUI for Container Administration

Changing default location of docker containers

Changing the location of docker named volumes

Remotely Administering the Docker Engine on Windows Server 2016

Building a container running SQL Server 2014 SP2

Automating installation of Docker & SQL command line tools on Linux

Limiting resources available to containers

Enabling the SQL Server Agent in Linux Containers

Default resource limits for Windows vs Linux containers in Docker Desktop

Using Windows stored credentials to connect to SQL in containers

Loopback available for Windows Containers

Setting up SQL Server replication in containers

Using the GitHub Package Registry to store container images

Updating SQL Server container memory limits on the fly

Running a SQL Server container from scratch

Using environment variable files for SQL Server in containers

Building a Docker image from a Github repository


Docker Compose

Creating SQL Server containers with Docker Compose

SQL Server & Docker Compose


Persisting Data Series

Persisting data in docker containers – Part One

Persisting data in docker containers – Part Two

Persisting data in docker containers – Part Three

Using docker named volumes to persist databases in SQL Server


Docker Swarm

Running SQL Server in Docker Swarm

Using Secrets in Docker Swarm for SQL Server

Persisting data for SQL Server on Docker Swarm with Portworx


Kubernetes

A shell for working with Kubernetes

Running SQL Server in Kubernetes on Azure Container Services – LEGACY

Running SQL Server in Kubernetes on Azure Kubernetes Services (AKS)

Exploring the Kubernetes dashboard in Azure Container Services

Configuring Persistent Storage for SQL Server in Azure Kubernetes Services

Changing the port for SQL Server in Azure Kubernetes Services

Azure Kubernetes Service Error – LoadBalancer type service external IP stays pending

Creating a custom kubectl plugin to connect to SQL Server in Kubernetes

Chaos engineering for SQL Server running on AKS using KubeInvaders

Converting SQL Server docker compose files for Kubernetes with Kompose

Use port forwarding to access SQL Server running in Kubernetes

Adjusting pod eviction time in Kubernetes

Differences between using a Load Balanced Service and an Ingress in Kubernetes

A storage failover issue with SQL Server on Kubernetes

Overcoming a storage failover issue with SQL Server on Kubernetes using Portworx


Helm

Deploying SQL Server to Kubernetes using Helm

Creating custom SQL Server Helm charts

Using Github as a repository for SQL Server Helm Charts

Decoding Helm Secrets

A kubectl plugin to decode secrets created by Helm


Azure

Pushing SQL Server images to the Azure Container Registry

Deploying Azure Container Instances

Running a SQL Server Azure Container Instance in a virtual network

Creating SQL images in Azure with ACR Build – Part One

Creating SQL images in Azure with ACR Build – Part Two


Terraform

Deploying SQL Server to an Azure Container Instance using Terraform – Part One

Deploying SQL Server to an Azure Container Instance using Terraform – Part Two


Raspberry Pi

Running Azure SQL Database Edge on a Raspberry Pi

Building a Raspberry Pi cluster to run Azure SQL Edge on Kubernetes

Updating my Kubernetes Raspberry Pi Cluster to containerd

Provisioning storage for Azure SQL Edge running on a Raspberry Pi Kubernetes cluster


External Articles

Running SQL Server Containers on Windows Server 2016 Core

Refreshing SQL Containers automatically using Watchtower

Streamlining QA with SQL Server Containers


Exporting Container Images

I’ve already covered how to push images to the Docker hub here but what if we only want to share a custom image in-house?

Well, it’s really simple to do by using the docker save and load commands. I’ll run through the process.

What I’ve done is created a custom image following the instructions here. Here’s my custom image: –

dockercustomimage

What I’m going to do now is save my custom image to a location [C:\temp] on my host server. So I’ll run: –

docker save -o myfirstimage.tar myfirstimage

N.B. – I’m saving the file as a .tar as this seems to be the format that works best for loading the image. I’ve tried exporting to other formats (.zip) but it always seems to become corrupt. The .tar format has always worked for me so I’ll stick with it.

Now the SQL images aren’t the smallest so this could take some time. But once it’s complete you’ll see the save .tar file: –

dockersavedimage

This can now be copied over to another server running the Docker engine and loaded into the local registry by running:-

docker load -i myfirstimage.tar

dockerloadimage

This will take some time but once it’s complete the exported image will be available in the server’s Docker registry. So that image was built on one server, exported and then loaded on another server. It’s an easy way to share images without having to push up to the Docker Hub.

By doing this you can ensure that every one of your team is using the same configured instance of SQL Server. Hopefully an end to “well it worked on my machine”…

Thanks for reading!