Setting up a CI for pclub.in

To begin with, I urge you to have a look at our Github repo for this website. Sure, Github serves the purpose of keeping our code public and accessible, and also lets users open issues and pull requests for the same.

But more, we wanted to use Github as a medium to build the website automatically on commits to the master, and deploy the website as well. This would mean, anyone can write a post, or fix a file using Github’s online interface, send a pull request; which can be merged online as well. At no point was there a need to download the repository, and yet, the website would build in less than 2 minutes, and be updated for public. Isn’t this cool? Hang on for how this was done.

Some basic information

A lesser known fact is that Github also allows you to host webpages. For instance, this website is also hosted on Github at pclubiitk.github.io/pclub.in. For this, you just need to keep your html files in a branch called gh-pages for a project repo, or in the master of a repo called username.github.io. This is itself a big topic to cover here.

An even lesser known fact is that you can use Github to build your static webpages for you automatically, using Continous Integration (CI). This is only true for Jekyll though.

The challenge

On deciding about setting up a CI for this website, I had several options in hand. But before that, we had several constraints as well. Some of them were:

The website uses custom gems. Github’s automatic page building would not build it.
We host this website on a server inside IIT Kanpur. That server is not directly visible to the outside world, and only the HTTPS port is forwarded. So an external CI cannot send its built files to the server directly.
Deploying a local CI seemed like an overkill. Plus, IIT Kanpur’s internally deployed Gitlab wasn’t fully ready to be able to handle builds etc yet.
Even if one could somehow build the files, it would not be a direct task to get the files inside IIT Kanpur.
Providing permission to an external CI to push to the repository would require committing the private key, which is insecure.

The resolution

We finally used Travis CI for this purpose. Travis is an online Continous Integration service, which offers to build your repositories for free, within some limits. You can customize how to build, and what else to do along with that much information. We set it up to build the code, and push it to the gh-pages branch of the same repo.

Giving travis push access to the repository

For travis to be able to push to the repo, it needed to be provided a ‘deploy-key’, a key which provides access to a repository to the person having the private counterpart of that deploy key. I generated a SSH-key pair, and added the public key to Github on this repository. As for the private, it had to be inside the repository. But then anyone could copy it and pretend to be our CI. The way out was this. Turns out, Travis developers thought of this, and provided you ways to encrypt and decrypt files without storing the password anywhere. So finally, I ended up committing an encrypted version of the deploy-key, and then decrypting it inside the build machine during building. Have a look at that article to know how it can be done.

Deploying to the branch

This was fairly easy. We make a folder named _site, and execute git init, git remote ... and git pull in that. We also checkout the gh-pages branch. This is followed by the build. So basically, the required files get updated inside the _site folder. We add all files to git, and commit. This is then pushed to the origin, and we have the up to date build files in the gh-pages branch.

Getting the files inside the server

Since the pclub server is running blind to the world, there was no direct way for it to know that the repository has been updated. webhooks came to the rescue here. So Github has another gem, which lets you execute arbitrary actions on events. So first, I enabled PHP on the server which is serving this website, followed by a randomly named PHP script, which checks for a valid request, and pulls the updated repository from Git.

Think of it as a way to tell the machine that the code has been updated online, and it should pull in the new code. So currently, on every commit, the server will pull in the updated website. This is very convenient, as you can imagine, and simplifies the task of maintaining the website considerably.

Of note, the original version of this Gist was a major help in all this.