Docker on the RPi

This is just a place holder for a future guide and hopefully useful information for someone who searches.

Docker is a very interesting virtualization technology and I’ve been wondering what can be squeezed into a Raspian OS and 1gb RPi 3b+. So I decided to build it – but it went wrong on Raspian Buster.

Why Raspian? Well its the community supported OS based on Debian for RPi. I have other Pi images I’m happy with running on Ubuntu but its a bit messy with some hardware options. So I decided to use Raspian. However the latest “Buster” distribution versions from late 2019 on. Since I had run into a bunch dpkg errors … i thought i’d work with the older “stretch” distro of Raspian. Not much older – I got the latest one from here and used Imager from RPi to write it to microSD

These are 2 pieces of information I used so far to install it via apt instead of via the downloaded script from docker (which itself calls apt to retrieve it). I wanted it to just install like any other package.

Get the sources right

sudo touch /etc/apt/sources.list.d/docker.list
### then add this line to that docker.list sources file
deb [arch=armhf] https://download.docker.com/linux/raspbian stretch stable

Documentation from Docker on how to install it via command line.

You should reboot before you get to the point of testing with

sudo docker run hello-world
##output from this
Unable to find image 'hello-world:latest' locally
latest: Pulling from library/hello-world
4ee5c797bcd7: Pull complete 
Digest: sha256:8e3114318a995a1ee497790535e7b88365222a21771ae7e53687ad76563e8e76
Status: Downloaded newer image for hello-world:latest

Hello from Docker!
This message shows that your installation appears to be working correctly.

To generate this message, Docker took the following steps:
 1. The Docker client contacted the Docker daemon.
 2. The Docker daemon pulled the "hello-world" image from the Docker Hub.
    (arm32v7)
 3. The Docker daemon created a new container from that image which runs the
    executable that produces the output you are currently reading.
 4. The Docker daemon streamed that output to the Docker client, which sent it
    to your terminal.

Its good to know that Docker Images have to be made for the RPi system architecture so you have to find RPi images. The “docker-apps designed for x86/x64 and i386 architecture on your Raspberry Pi will return an error”.

Learn admin on an RPi? Part2 – Reverse Proxy

So I’m not the only one who thinks admin is all about the command line, the terminal – the ssh session and so on. SSH makes you as powerful as the guy asking Clyde to punch problematic bikers in Any Which Way but Loose with Clint Eastwood (which you might want to find and watch using your torrent seedbox … maybe someone recorded it to a file and its not copyrighted … probably not?

A Word on Copyright and Torrents

That’s why its a grey area – we used to record stuff from TV to VHS tapes and give to each other or leave laying about in community libraries. These days people want to seek rent on every view … not everyone agrees – it could be illegal (copyright law). So don’t assume, DYOR and I live in a country where its more fuzzy interpreting these laws somewhere in the east. Some law is just about how much is it worth for them to pursue and enforce copyright. Other times we have fair use – but law is not my expertise. If you sell copies – well that’s definitely more illegal than just accidentally viewing a screen without paying someone pay per view fees. Is this GIF meme illegal – some days it seems to be infringing copyright, other days its just fair use?!

On one hand we can, if it all works find our way around graphical tools that label options with buttons and check-boxes rather than expect us to read and know how to construct a command. On the other hand we tend to just stay flat and not move up the learning curve that leads to confidence. The command line and the .conf file are the reality to some … to others they are the same, just different approaches. In Linux however the command line is most complete.

In Part 1 – i linked to the Raspberry Pi (RPi as we write in shorthand) and explained that its a great, affordable way to have a server of that you can install, take with you or totally expand upon. Its as powerful as many basic level virtual servers you might rent online to build your website projects and other infrastructure. So its basically a direct match, you can put linux on it and there are projects.

Visit Part 1 if you haven’t built a project that you would want to follow up with these proxy, sharing ideas. Much of that was about basics of getting started and lead into the project – if you have an RPi up and running you could follow this guide as mentioned in Part 1.

The seedbox project is the culmination of understanding the need for privacy when it comes to using bittorrent. Many people use it for getting files that are in the grey (or red zone) when it comes to copyright infringement – but its also just a tool for all of us to share files or download in a nice resilient, time-saving way. I can throw the torrent files at the seedbox web interface daily and sftp them locally into my main work PC, or my home PC. My server is effectively online accessible where-ever I travel – when I have a “hosted seedbox”.

What if its an RPi operating from your home. Well, first – how do you access it at all if your ISP is a private network and you have no way to forward from your public IP address because it simply doesn’t start at your router, its further out there in the ISP and you can’t request forwarding rules. They have decided you can’t host … that way. So the answer – reverse proxy and/or tunnelling.

So here is what is way faster about admin on the command line – installing software. You need to learn sudo and apt and then you just read what else to type to install packages to enable you to follow guides and make things happen. Snap (or snapd) is another package system that you might use to speed up installing ngrok. It tends to be in ubuntu by default these days. This is a way unique to linux.

sudo snap install ngrok

When I installed to my Ubuntu 19.1 PC the snap install confirmed with “ngrok 2.3.35 from Khiem Doan (khiemdoan) installed”

The other way to install ngrok (if you don’t want to use snap or don’t have it) is to download the file from the ngrok website and then … unzip it into your home directory or where-ever you decide you want to organize the files. This is common for Mac, Windows and Linux. Don’t take “/path/to” literally it refers to where you put ngrok.zip

unzip /path/to/ngrok.zip

The Ngrok tool – assuming it is downloaded or installed via snap – has a server side. What you installed was the client side that will open a tunnel through the server. Ngrok have free accounts on their server. So ngrok is a little mystery for you to solve but its all the more clear when make an account with https://ngrok.com/ – then you get yourself a authentication code as you follow their getting started guide. I operated that with the local server port via https as so:

The getting started guide will explain that you should run ngrok with the authtoken command … note it depends how you installed ngrok if you are running it from the directory you extracted it to via “./ngrok” or simply typing ngrok because its been properly installed into /usr/bin – you can also just copy ngrok there yourself. The command generates a config file … but you can also just edit/create the file … its up to you to read the guide.

$./ngrok authtoken 1ifj49fjls43jl49090f90590seg
Authtoken saved to configuration file: /home/owner/.ngrok2/ngrok.yml

./ngrok https 8112

Then you get to see ngrok connect to the server and if you have set it up as per their guide you’ll get a URL. Here is one I just ran as an example – it will be destroyed after I close ngrok. You can see I have an account “ja” – thats find. I don’t mind if you see that. I don’t use this any more – as I’ll explain – because I switched to hosting my own ngrok server.

So in testing this with deluge on port 8112 I was able to give access to the deluge-web gui – a website running on the Rpi. I also configured sftp so my friends could get to where the files were seeded or downloaded to and retrieve (anonymously) the files. sftp is encrypted – so its just a private encrypted data stream from their computer to yours. Even if you run an http session you can https expose it with ngrok which gives it a real certificate and encrypts the website. These days everything should be encrypted or it’s a security risk. My friends maxed out at about 8-10 mbit download speeds, but my home has 150mbit fibre up/down speeds. Which I wanted to leverage. So you could either

1. Pay Ngrok for a plan that has more options/power. It doesn’t really say what kind of throughput you would get. This may not be appropriate for seedbox sharing …
2. DIY Ngrokd (note that adding a d (daemon) letter after a program tends to refer to a process running as a server – staying alive, waiting for connections internally or over the network. For this one you need a server.
3. Find a different solution for reverse proxy and there are many ways to skin that cat. You could just open tunnels if you know how it is done. Here is someone taking time to explain what goes on inside ngrok and other tools – you could do that yourself and its on my todo list for “someday”.

Option 2 – DIY Ngrokd (host your own)

So I have a server that does very little – it has mail services that I built as part of another project. Very basic set up that was. Its mostly on idle but it has 2gb RAM and 5TB of monthly bandwidth … but that company is complex. I also use this Vultr.com hosting company more recently – you can get $100 free credit if you sign up and pay like $10. The truth is you’ll never really get away with anything quality for less than $10 sign up – so don’t spend too much time looking for free virtual servers – it costs you in the end.

I definitely recommend Vultr and if you are learning admin you could start up a lot of stuff in your first month on that credit and pay effectively nothing for experimenting. Vultr also have a lot of free extra services – unlike some more mercenary hosting companies that want you to pay for every little extra. You can become an admin boss just by installing VPS and using applications that accelerate everything such as Plesk – so rather than choose an OS, look at Application. Most of them are preset to install an image with Ubuntu 18.04 x64. If you are really learning you can just start from a generic Ubuntu and install it yourself but … it might take you down a lot of pathways where things aren’t tuned right and bugs or failures need to be fixed one after another before things just “work”. That is why applications are already mostly set up to work well and you get to the results as fast as possible. Depends how much time you have to invest – both ways are good. I’ve installed servers every which way I can because – its just like that doing admin for years. Don’t be afraid to discard and reset to a fresh one if its all going wrong.

If you want to keep going I’ll assume you have a server … say ubuntu 18.04 type … you can build the ngrokd service. You’ll get the ssh username/key. Don’t use crap passwords .. you’ll very easily get hacked. This guide comes from the maker but its an older version … its a variation and doesn’t use any authkey – but this self hosting, its a challenge. On an RPi you’ll – you might find that when you build both the client and the server in this walk through that go doesn’t work. Rpi have a different CPU so for it to work that was something extra I had to do because Ngrok is using go or “golang” https://golang.org/

So this was the Reverse Proxy escape hatch

Solution to having an ISP “private network” and no usable IP address is reverse proxy. If you made your own server and put ngrok on it – congrats. However if you want your torrent application itself to be anonymous you aren’t there yet. If you get a torrent then its possible the tracker gives your IP away as it makes contacts with other seed/leechers of a torrent file your getting. Even if you use a magnet link and DHT – that still links you to trackers that are open … so you need to anonymize with a proxy for the send/receive torrent data. I highly recommend IPvanish – its got the highest throughput of anything I’ve tested. Recently I noticed they also give a 250gb cloud storage that is actually fast and usable for online backup. I connected it to rclone and uploaded backups of my RPi sd card made with dd or imagewriter32. That’s $99 a year of free cloud storage. I can tell you … I’ve tested a lot of cloudstorage companies and many of them are a total joke so I was enormously chuffed. So if you want to be able to follow through in the next part go ahead and sign up for IPVanish – its usually 30 days free (money back guarantee) so you can just cancel it after you’ve played around and don’t want to keep it. Here is a tip – if you idle for a while on the signup page … a popup will come and offer you an extra 20% more discount! [I recently changed the link to their super discount deal].

I’ll get into more of the admin problems and solutions- like proxy, security, firewall, stability and crontab scripts, monitoring your connection so you can restart the ngrok or reboot the computer if its stopped functioning in Part 3 which is not written yet but will come soon.