📰 Good News app. Backend in Golang. Docker Compose & Traefik v2.0 on Digital Ocean.

Let’s show our project to the world!

I am glad to see you on the last article out of six in this chapter.

If you are not familiar with what I am going to implement in this series of chapters, I would recommend you to read the introductory article:

📰 Good News app. Golang + Flutter + Hummingbird = 🖤

All chapters of a “book”:

1. 📰 Good News app. Backend in Golang behind Traefik reverse proxy with https available.
2. [in progress] 📰 Good News app. Flutter for rapid mobile applications development.
3. [in progress] 📰 Good News app. Hummingbird as a promising replacement for frontend frameworks.

And here are articles of current chapter:

1. Prerequisites & Idea, project and database structure and API endpoints.
2. Project creation, go modules & GIN (beautiful framework) integration.
3. Colly usage.
4. MongoDB setup using official Golang driver.
5. Running all together locally with Docker and Docker Compose & Traefik v2.0 configuration.
6. Publishing to Digital Ocean, Let’s Encrypt and DNS Challenge configuration.

So let’s start!

This article can be used as a separate tutorial for deploying your backend on Digital Ocean (DO) with https configuration using Traefik and setting it all up for your own domain.

In this article we are going to register a domain on Domains.Google (or any other provider, all settings will be intuitive to set up), change DNS records to those which are provided by DO, create a droplet and deploy backend there, set up Traefik to enable https requests to our server by gathering certificate from Let’s Encrypt.

In order to fully and successfully complete this article, you will need a DO account and some money to buy a domain. I do not call you to do it, you can just make a cup of coffee and see how I do it and when you will really need to make all those steps for your own project, you will open this article and follow all the steps. In addition I would like to repeat my words from first article where I say that this project is already available on my domain and if you would like to check it or use it for future articles where we are going to build mobile apps, please feel free to do it.

Registering a domain on Domains.Google

You can choose any provider for registering your own domain, for example, GoDaddy or others, however I prefer Domains.Google for this purpose. If you do so as well, you will need to login, then choose a domain you desire and purchase it. Then you will see a purchased domain in the list of all domains, click on it and now we will set up DNS records so it redirects to DO.

So choose DNS on the menu on the left and then click on something that says Use your own DNS-servers and then add three DNS records which point to DO DNS servers.

ns1.digitalocean.com
ns2.digitalocean.com
ns3.digitalocean.com

I think you will find almost the same settings on other providers’ sites where you can register your domain. All providers always say that changes to DNS records will be applied during 24 hours.

That’s it for registering a domain. Now we will move to DO dashboard.

Digital Ocean

So now please login to Digital Ocean. Once you are on the dashboard page, navigate to Manage → Networking on the menu on the left. Go to Domains tab and you will see that you can add a domain, so enter your domain to the input field and press Add Domain button.

Once you add a new domain, it will open a page with settings where you can add other records which could be used to set up a mailing service or other useful things. So and in the end of that page you will find three DNS records with type NS. They are exactly the same that we have put to Domains.Google. We will come back to that page when we will redirect all requests for api.good-news.your-domain.com to the droplet (dedicated server) where we will deploy our backend.

Now let’s create a droplet and deploy our backend! But wait, in order to have access to a remote droplet or server on DO, we need to generate SSH key on our machine and that SSH key to your profile. So DO will know that exactly you connect to their servers and will let you manage your droplet through the terminal.

So now go to Account → Security on the menu on the left. Then press Add SSH Key button and you will see very handful tutorial on the right how to do it. Please follow all the steps and create one, we will choose it once we create a droplet.

Once you have done it, you should see big green Create button on the top of the page. Press on it and pick Droplets. You will be navigated to the page where we need to pick settings for our future droplet.

So you will need to pick some options for the creation:

• Choose image: Go to Marketplace tab and choose Docker image from the list. It means that Docker will be preinstalled on a server.
• Choose a plan: For educational purposes, let’s pick the cheapest option that costs $5 a month.
• Add block storage: None.
• Choose a datacenter region: Well it depends where you live, I usually pick one in Europe.
• Select additional options: None.
• Authentication: Here pick a SHH Key that you have added earlier.
• Other options are up to you.

Then press Create Droplet button and wait for a few minutes while DO is in process of creating your droplet.

Once your droplet is created, please go to Manage → Networking → Your Domain and create new record as it is filled up in the picture below.

As you can see for the Hostname we set a subdomain api.good-news and it will understand that it should be applied to your-domain.com. Then it should redirect to the droplet that was created earlier. Then press Create Record and you will see that a new record with type A was created. It means that all requests to api.good-news.your-domain.com will be redirected to the droplet where we will deploy our backend soon.

Deploying backend to DO droplet

So now we have to establish a connection to our droplet through the SSH in the terminal on our machine. Before doing this, please copy IP address of your droplet, you will need it later. Open a terminal window and type

> ssh root@droplet_IP

It might ask you to add this IP to the list of trusted IPs, please confirm everything. Then you will be asked for a passcode that you might have set while creating a SSH Key. After that you have to be connected to your droplet. In order to check, type

> ls -l

It won’t show anything because our droplet is empty. I have the following folders structure on the server.

acme/
    acme.json          - should be here for generating certificates for https
appdata/               - is used by MongoDB, no need to create it by our own
good-news/
    api/               - where our project is going to be located
    docker-compose.yml - Docker Compose file where all information for deploying is placed

So in order to create acme/acme.json, please type the expression below while being in the root folder of the server. And then we give permission 600 to acme/acme.json and create good-news/ folder.

> mkdir acme && touch acme/acme.json
> chmod 600 acme/acme.json
> mkdir good-news

Then move to good-news/ folder because all the rest manipulations will be done in this folder.

> cd good-news

So now we have to clone our project repository from Github, create docker-compose.yml and .env files and fill them out. For the first step you could use my repository, I promise there is no difference with what we have been doing during previous articles. Or you could do it with your repository but don’t forget to put vendor/ folder and other unnecessary files to .gitignore.

> git clone https://github.com/kanzitelli/good-news-backend api/

It will clone all the code from the repository to api/ folder.

Let’s create .env file and fill it with needed information.

> nano .env

So for DOMAIN_NAME please use your domain with good-news subdomain. For my case it is good-news.ggc.team. You might have noticed usage of new variable DO_AUTH_TOKEN. It will be used by Traefik to make DNS Challenge right. I am not big expert how it all works but you get that token from your DO dashboard by navigating to Manage → API → Personal tokens and generating a new one. Copy and paste it to .env. Then we disable debug mode for our backend so it will run in release mode. And we provide some variables for setting up MongoDB. You can use your credentials.

In order to close file editing, press Ctrl + X and then to save it type Y (for YES) and enter.

It is time to create docker-compose.yml file and configure Traefik. This was the most painful part when I was doing it for the first time. Also I would like to warn you that as we use Traefik v2.0 and it is not in fully production mode, they are testing it, something might be broken. I have faced it once while using v2.0 but reading their docs for 30 minutes solved my problem. Hopefully this is going to be final and working variant. So create docker-compose.yml file in the ~/good-news/ folder and paste the code below.

> nano docker-compose.yml

Please, make sure to change certificatesresolvers.mydnschallenge.acme.email to the actual email and domain has to be the same as your registered domain.

So for reverse-proxy (Traefik) service, we provide more information comparing to what we have done when we have launched it on local machine. So we set ports for our http and https entry points to :80 and :443 respectively. Then we set dnschallenge to true and set that we want it from digitalocean. Then we set email so Let’s Encrypt knows which email to use when generating a certificate and we set a path for acme storage file that we have created earlier. We provide DO_AUTH_TOKEN to the environment variables for Traefik taken from .env file. In ports part everything should be clear, we use :80 port for http and :443 port for https. And in volumes we provide a path to acme file that is located on our server machine.

For api (Golang backend) service, we have almost the same properties as we have defined in the previous article except of extra labels. So in those labels, we set a Host to api.${DOMAIN_NAME} and DOMAIN_NAME is taken from .env file, so Traefik will know where to direct requests. We use a middleware that redirects all http requests to https. Then we set a certificate resolver to one that we have defined in reverse-proxy service.

And for mongo service, I guess, everything is intuitive and the same as we have defined in the previous article.

Let’s run it!

It is time to run it! So as may guess we will face the same problem with setting up a MongoDB as we had in the previous article. I am going to put all the steps here as well.

> docker-compose build && docker-compose up -d

Wait for some time till whole process is finished. Type docker ps to ensure that all of the containers are launched and running successfully.

We need to configure MongoDB. Please run all commands below one by one without # comments. All credentials for MongoDB should be the same with those which are located in .env file in the root of the project (in case if you have changed them).

# Opening shell of mongo db running within our docker container
> docker exec -it mongo mongo -u "GGCTeamBatr" -p "MySuperSecretPassword" --authenticationDatabase admin
# Changing to (creating) a needed db
> use good_news_db
# Creating a super user
> db.createUser({user: 'suuuper_user', pwd: 'soop3r_U$eR_PSWD', roles:[{role:'dbOwner', db:'good_news_db'}]})
# Inserting test data to test collection
> db.test_collection.insert({ test: "test" })
# Displaying all collections of our previously created db in order to make sure that our test collection was successfully created
> show collections
# Saying goodbye to mongo shell
> exit

So now we have run docker ps to see all running docker containers and find CONTAINER ID (first column) of container named api. Then stop that container by typing > docker stop <api_contrainer_id> and instead of <api_contrainer_id>put the value found under CONTAINER ID. Then build and run the containers one more time to rebuild our docker container with new settings applied to MongoDB before.

> docker-compose build && docker-compose up -d

So it may take some time to generate a certificate for your domain. In order to check it type > cat ~/acme/acme.json and if there is any data it means that everything is fine. Otherwise try checking Traefik container’s logs by typing > docker logs 22tail 200 <traefik_container_id>, you can get traefik_container_id by typing > docker ps and checking the first column.

And once it’s done you can open a browser window and go to https://api.good-news.your-domain.com/v1/news/sources. Now you should see news sources which are pre-filled before server launch. After 3 minutes, you will be able to see first news gathered from parsing news sites by opening https://api.good-news.your-domain.com/v1/news.

So this is the end of the first chapter. We have launched all containers successfully on Digital Ocean with https available 🥳 Thank you for reading till here! Below you will find the link to the first article of the second chapter where we will be creating mobile apps using Flutter.

It is not ready yet but will be available as soon as possible. I am working on it now 😉

If you have any comments or suggestions, please feel free to write them down or email me at batr@ggc.team 🙂

If you would like to know when I post new articles, follow me on twitter 🐦

ʙ ᴀ ᴛ ʀ