View source for Sysadmin Documentation/SSL

'''this is a work in progress'''

In order to get SSL certs for free and get rid of those pesky warnings, we must take advantage of the service provided by letsencrypt.  Our solution that uses letsencrypt is chosen to be simple to set up, while also being the least invasive to the other servers.  This minimizes any potential maintenance issues and also allows easy transition to either a paid ssl service or a different free SSL service if one appears.

==Overview==
We use the letsencrypt DNS challenge option with a client located on a dedicated UGCS server.  The certs are periodically refreshed on this server, then copied over to the relevant UGCS servers.  

Q: Why don't you just install certbot on each of the servers?

A: First, because there's lots of servers and it would be a pain to install and update on all of them.  Second, because it pulls lots of gay deps that we don't really want shitting up the server.  Third, since they require integration with the server, we will need to worry about integration with the server software like apache, and we'll also be subject to any third party fuckups which could then compromise our services.

tl;dr, because certbot shits the place up too much

==letsencrypt client==
We are using the letsencrypt.sh client, since it doesn't pull in too many gay deps and is really simple to work with, and it's not gay like certbot.  In order to do dns integration with route53, we download this random script "lets-encrypt-route53.rb".  Yeah I know that's really frickin sketchtacular but it works and that's all we really need. This means we also need ruby and the aws ruby sdk (which is official, unlike the python one, so yeah).  make sure to 7 the rb file

throw some things in the domains.txt file
 ugcs.caltech.edu wiki.ugcs.caltech.edu etc...

so we export the env variables

 $ export AWS_REGION="us-west-2"
 $ export AWS_ACCESS_KEY_ID=""
 $ export AWS_SECRET_ACCESS_KEY=""


then run that shit.
 ./letsencrypt.sh --cron -t dns-01 --hook ./lets-encrypt-route53.rb

Now we have a cert congrats

==Automatic renewal and installation==
Letsencrypt has certificate lifetimes that are short enough that updating certificates manually will be a pain in the ass.  This means we need a system to update the certificates completely automagically.

We also don't want to compromise the availability of the system at all or introduce any additional points of failure.  An expired certificate is tolerable, but a broken system that makes the certificate completely unavailable will cause complete loss of all services.  The only viable solution to this is to install the certificates directly onto all of the local drives of the servers.

=== Certificate Renewal ===
In order to renew the certificates, we use cron.  We run the previously noted command using the crontab to slurp the certs.  Since the script will only touch certs soon to expired, we can just run it all the time, like daily.  This means we won't worry about having to sync up the cert updates or anything like thta

=== Certificate Installation ===
Since we'll have services that need to be restarted when the certs are updated, it's more convenient to have each server pull the cert from the cert updater computer.  We have a cronjob that will use SCP to download the certificate, diff it with the old one, then copy it to the proper place for the cert to be.  The cron script will then call systemd to restart the services that use the cert if the cert has been changed.

The script is here for your convenience. right now we run it monday at 4am, but that's arbitrary
 cd /root/ssl-updater
 
 rm -rf certs-old
 mv certs certs-old
 mkdir certs
 
 edited=0
 
 scp [redacted]/privkey.pem certs/
 scp [redacted]/fullchain.pem certs/
 
 diff certs/privkey.pem certs-old/privkey.pem
 if [ ! $? -eq 0 ]
 then
 
 cp certs/privkey.pem /etc/ssl/private/ugcs.key
 
 edited=1
 
 fi
 
 
 diff certs/fullchain.pem certs-old/fullchain.pem
 if [ ! $? -eq 0 ]
 then
 
 cp certs/fullchain.pem /etc/ssl/certs/ugcs.pem
 
 edited=1
 
 fi
 
 if [ $edited -eq 1 ]
 then
 
 service whatever restart
 
 fi



Q: Why don't you have each server renew its own certs

A: We'll need to throw the AWS route53 credentials on each server, which is kinda shitty since if any server gets pwned, then the dns is dead and then mitm and we're dead to death beyond dead