In preparation for remote teaching this year, I’ve spent the last few weeks converting the Doctoral Training Centre’s ‘Introduction to Computer Programming’ course into a series of Jupyter notebooks so that the course can be run entirely using Google Colaboratory.
Continue readingCategory Archives: How To
How to SCP files from a gated server to your personal computer
Jack recently made a blog post in which he provided a script which can transfer your files between your personal computer and a given remote machine via temporarily hosting a file on file.io (blog post here); where you have some sensitive data that you do not want to risk hosting online, you can also fairly easily use SCP to keep business strictly between your local and remote machine.
What I am referring to is described here. This blog post refers to the case where you want to SCP from local host to a remote machine which is only accessible via a gate server (this isn’t necessarily true for the Stats computers as we can use the VPN to directly access our remote machine of choice by the way). I won’t effectively plagiarise the blog post I linked to as the explanation is clear enough in itself, but you just use port forwarding and the localhost address of your local machine!
Best wishes,
Eve
Prerecording Conference Talks and Posters using OBS Studio
Seemingly every conference due to take place this year has either been cancelled or will be run virtually due to the COVID-19 pandemic. Many organisers have decided that running entirely live virtual programmes causes more trouble than it’s worth (e.g. due to unforseeable IT and internet issues disrupting the schedule), and so are asking their presenters to prerecord their talks, which are then broadcast “live” on the day.
I recently “presented” two virtual prerecorded talks at the ISMB conference using Open Broadcast Software Studio (OBS Studio), a free open-source software package most commonly used by live-streamers on Twitch and Youtube. It is super simple to use and achieves a professional output, with video overlaying a presentation slide deck/poster PDF. This blog is a “how-to” on getting started with OBS for conference talks/poster presentations.
Continue readingUploading/downloading small files across systems
Sometimes you just want to quickly move a copy of a script, image or binary from, for example, your local (linux) machine to another (linux) machine. The usual tool would be SCP, but this can get complicated when there are several layers of ssh and sometimes it doesn’t work at all (as is the case for transfers between the Department of Statistics computers and the outside world).
Continue readingStreamlining SSH for remote work
With the university now working remotely, and our group working entirely on linux systems, I figured that now would be a good time to share some useful SSH commands to streamline remote access. This is far from an exhaustive list, but will hopefully serve as a useful starting point for anybody who finds themself needing to work remotely on a linux system.
Continue readingUsing SLURM a little bit more efficiently
Your research group slurmified their servers? You basically have two options now.
Either you install all your necessary things on one of the slurm nodes within an interactive session, e.g.:
srun -p funkyserver-debug --pty --nodes=1 --ntasks-per-node=1 -t 00:10:00 --wait=0 /bin/bashand always specify this node by adding the ‘#SBATCH –nodelist=funkyserver.cpu.do.work’ line to your sbatch scripts or you set up some template scripts that will help you to install all your requirements on multiple nodes so you can enjoy the benefits of the slurm system.
Here is how I did it; comments and suggestions welcome!
Step 1: Create an sbatch template file (e.g. sbatch_job_on_server.template_sh) on the submission node that does what you want. In the ‘#SBATCH –partition’ or ‘–nodelist’ lines use a placeholder, e.g. ‘<server>’, instead of funkyserver.
For example, for installing the same conda environment on all nodes that you want to work on:
Continue readingConsistent plotting with ggplot
Unlike other OPIGlets (looking at you, Claire), I have neither the skill nor the patience to make good figures from scratch. And making good figures — as well as remaking, rescaling and adapting them — is incredibly important, because they play a huge role in the way we communicate our research. So how does an aesthetically impaired DPhil student do her plotting?
Continue readingA few more reasons why UNIX is awesome
One could easily find dozens of reasons for which UNIX — mainly Ubuntu — is simply, the best operating system. Although I remember people in my proximity mentioning this for ages, it’s been only a few months that I’ve realized what are the true advantages. Helpful for this were all the people teaching/demonstrating in various modules during my first year in SABS/DTC: quite often we would be asked to do something in the console rather than by clicking the mouse. In the meanwhile, I’d wonder why using the console can be better from a nice, user-friendly GUI (i.e. Windows…). Tools like sed, grep, tar and of course alias-ing form a quick answer. I will not argue more about these but demonstrate two more tools/tricks.
Continue readingCombining Inset Plots with Facets using ggplot2
I recently spent some time working out how to include mini inset plots within ggplot2 facets, and I thought I would share my code in case anyone else wants to achieve a similar thing. The resulting plot looks something like this:
Python Handout
Many OPIGlets extensively use Jupyter (in either Notebook or Lab flavour) to prototype and present their work. However, as project progress frequently notebooks are converted into regular python files for a number of reasons, losing the notebook functionality.
Wouldn’t it be nice if we could combine some of the benefits of Jupyter notebooks (not least the ability to present both code & results naturally) with regular python files?
Enter Python Handout.
Python Handout was recently (5th August 2019) released by Danijar Hafner and allows Python scripts to be converted into handouts with Markdown comments and inline figures (see above picture).
Installation is via pip (pip3 install -U handout) and Python Handout supports python 3 scripts.
While I’ve not used Handout much (yet), I will definitely be experimenting more in the coming weeks.