In this blog post I will introduce a fun R plotting function, ggpairs, that’s useful for exploring distributions and correlations.
Category Archives: Code
Should scientists learn C++?
Conventional wisdom dictates that compiled languages are slow to develop, can be slow to compile, but are fast to run. Interpreted languages are easy to use and do not require compilation but have sluggish performance. Like most people in scientific computing, the first two languages I learned were C++ and Python; I use Python every day but when, if ever, would I use C++?
Continue readingQuick Python tricks
It’s always fun when you stumble across something in your programming toolkit that you had never noticed. Here are three things I’ve recently enjoyed learning.
- Ternary syntax
a = int(raw_input()) is_even = True if a % 0 == 0 else False
- Enumerate
I’ve been looping over the length of my list, all these years, like a chump. It turns out you can do this:
for index, item in enumerate(some_list): # now the index of each item is available as well as the item
# Don't do do this for index in range(len(some_list)): item = some_list[index]
- for… else
Every so often, you really need to know that a for loop has run to completion. That’s what for…else is for!
for item in iterable:
if item % 0 == 0:
first_even_number = item
else:
raise ValueError('No even numbers')
Property based testing in Python with Hypothesis : how to break your own code before someone else does
Traceback (most recent call last):
ZeroDivisionError: integer division or modulo by 0
We’ve all been there. You’ve written your code, tested it out on some toy data and then when you make the move to the real data, there was something you didn’t expect.
Maybe some samples have been truncated to zero. Maybe the input arrays are the wrong shape. Suddenly your code comes crashing down around you, and you’re left thinking: well how could I have known that was going to happen? I can’t test everything
Continue readingSome useful tools
For my blog post this week, I thought I would share, as the title suggests, a small collection of tools and packages that I found to make my work a bit easier over the last few months (mainly python based). I might add to this list as I find new tools that I think deserve a shout-out.
Biopandas
Reading in .pdb files for processing and writing your own parser (while being a good exercise to familiarize yourself with the format) is a pain and clutters your code with boilerplate.
Luckily for us, Sebastian Raschka has written a neat package called biopandas [1] which enables quick I/O of .pdb files via the pandas DataFrame class.
Continue readingMaking the most of your CPUs when using python
Over the last decade, single-threaded CPU performance has begun to plateau, whilst the number of logical cores has been increasing exponentially.
Like it or loathe it, for the last few years, python has featured as one of the top ten most popular languages [tiobe / PYPL]. That being said however, python has an issue which makes life harder for the user wanting to take advantage of this parallelism windfall. That issue is called the GIL (Global Interpreter Lock). The GIL can be thought of as the conch shell from Lord of the Flies. You have to hold the conch (GIL) for your thread to be computed. With only one conch, no matter how beautifully written and multithreaded your code, there will still only be one thread will be executed at any point in time.
Automated testing with doctest
One of the ways to make your code more robust to unexpected input is to develop with boundary cases in your mind. Test-driven code development begins with writing a set of unit tests for each class. These tests often includes normal and extreme use cases. Thanks to packages like doctest
for Python, Mocha and Jasmine for Javascript etc., we can write and test codes with an easy format. In this blog post, I will present a short example of how to get started with doctest in Python. N.B. doctest
is best suited for small tests with a few scripts. If you would like to run a system testing, look for some other packages!
Vim and I
Vim is great. Despite its steep learning curve , it has many advantages and many loyal Vim followers will tell you that you should force yourself to use it.
Personally I started using Vim when I was ssh-ing into the group servers or into my computer in department. In such scenarios, I could not open the IDEs with the nice GUIs 🙁 However, as time passed, Vim started to grow on me. Now, I can list a few reasons why I think it is great, for example, it requires a small amount of memory to run, has a short start up time and can handle large files pretty well.
Although, I am definitely not a Vim expert, I will tell you about some of the things I have added to my .vimrc
. The .vimrc
file is very handy for containing all your favourite settings, such as key mappings, custom commands, formatting and syntax highlighting. The file uses vimscript which is a programming language in itself. However, there is a lot of help online that tells you with what lines to add to your .vimrc
. I would recommend installing Vundle which is a Vim plugin manager.
Here I will list some cool things that I have discovered you can do with your .vimrc
. It has certainly made my life a bit nicer.
- Code Folding
Most IDEs provide a way to collapse functions and classes that results in only seeing the function/class definition and hiding the code. To do this in Vim add the following lines to your.
vimrc
" Enable folding set foldmethod=indent set foldlevel=99 " Enable folding with the spacebar nnoremap <space> za
Alternatively, you can install the Vim plugin SimpylFold. - Python indentation
Vim does not do auto indention like many IDEs. To automatically do PEP-8 indentation for Python, add the following to your .vimrc .
" PEP indentation au BufNewFile,BufRead *.py \ set tabstop=4 \ set softtabstop=4 \ set shiftwidth=4 \ set textwidth=79 \ set expandtab \ set autoindent \ set fileformat=unix
You can also install the Vim plugin vim-flake8 which is a static syntax and style checker for Python source code. It shows errors in a quickfix window and lets you jump to their location inside your code.
- Turn line numbers on
Rather than typing in:set nu
every time you open your files. You can always have them turned on by adding :set nu to your.
vimrc
- Autocompletion
When I switch from PyCharm to Vim I feel a bit lost without the autocompletion however, after a quick search I found many are using the Vim package Youcompleteme and it is awesome.
docopt for dummies
Parsing command line arguments is an annoying piece of boilerplate we all have to do. Documenting our code is either an absolutely essential part of software engineering, or a frivolous waste of research time, depending on who you ask. But what if I told you that we can combine the two? That you can handle your argument parsing simply by documenting how your code works? Well, the dream is now reality. Continue reading
Introduction to R Markdown
Two of our esteemed OPIGlets presented a workshop on collaborative research using Jupyter Notebook this week at ISMB in Chicago. Their workshop highlights the importance of finding ways to share your work conveniently and reproducibly. So on a related note, I thought I would share a brief introduction to another useful tool, R Markdown with RStudio, which I use to present updates to various supervisors and to remember what I did three months (or three days) ago. This method of sharing work is highly readable, reproducible, and narrative-driven.
I use R for much of my data analysis and all of my visualisation, and I count the tidyverse among my most beloved friends. If you’re so inclined, it’s easy to execute python, bash, and more from within R Markdown. You also don’t need to use RStudio to use R Markdown, but that’s a whole other story.
Starting a new markdown file in RStudio will generate a template script explaining most of what you need to know. If I showed you that then I’d be out of a blog post, but I will at least link to the R Markdown Reference Guide.
R Markdown files consist of text written in markdown, and code chunks that can be individually executed and displayed inline within RStudio. To “knit” the whole thing together, the knitr package is used to execute and combine code chunks, then pandoc converts the whole thing into an attractive document.
Here’s an example. The metadata at the top sets up the document. I’ll be generating an html document here, but notice some other tempting examples commented out. Yes, you can use it for Latex (swoon). You can even make a Word document, but really, why would you?
--- title: "Informative Title" author: "Clare E. West" date: "10/07/2018" output: html_document #output: beamer_presentation #output: pdf_document --- ```{r setup, include=FALSE} knitr::opts_chunk$set(echo = TRUE) library(knitr) library(ggplot2) library(tidyr) library(dplyr) ``` ## Big Title ### Smaller title R Markdown scripts have the extension .Rmd R Markdown is __so__ *fun*. You can read all about it [here](https://www.rstudio.com/wp-content/uploads/2015/03/rmarkdown-reference.pdf). ```{r} print("Hello world") ```
Notice that chunks are enclosed within three backticks, with the language and options in braces. Single commands can be executed inline using single backticks.
As highlighted in the example above, global options are set like this:
knitr::opts_chunk$set(echo = TRUE)
“echo=TRUE” means that the code in each chunk is displayed in the final product; this is useful to show collaborators (or your future self) exactly how you did something. Change this option (“echo = FALSE”) globally or in individual blocks to prevent code from printing. This is useful to hide uninteresting commands, or when presenting to people who don’t have the time or inclination to read your code (hard to imagine). Notice I’ve also used “include = FALSE” for the library-loading code chunk, which means evaluate but don’t include in the output. Another useful option is “eval = FALSE”, which means don’t even run this chunk.
So let’s see what that looks like when we render it:
Plots generated in code chunks or images from other sources can be embedded. Set the width in the options. “fig.width” sets the width (in inches) of the figure generated, while “out.width” scales the image in the final documents, for which the units will depend on the document type. Within RStudio, these are previewed inline below the code chunk.
## Including plots/images ```{r fig.width = 4, fig.height = 3, out.width = "400px", echo=FALSE} t %>% group_by(Tour, Winner, N, Tournament) %>% filter(WRank <= 20) %>% summarise(WPts = max(WPts)) %>% ggplot(aes(x=N, y=WPts, group=Winner, colour=(Winner=="Murray A."))) + geom_point() + geom_line() + labs(x="Tournament Number",y="Ranking Points") + scale_colour_discrete("",labels=c("Not Andy Murray", "Andy Murray")) + theme_bw() + theme(legend.position = "bottom", legend.margin = margin(0, 0, 0, 0)) knitr::include_graphics("https://s.yimg.com/ny/api/res/1.2/69ZUzNSMYb09GKd8CNJeew--~A/YXBwaWQ9aGlnaGxhbmRlcjtzbT0xO3c9ODAwO2g9NjAw/http://media.zenfs.com/en_us/News/afp.com/0102e1f7d0d3c35303c8a62d56a5eb79c2c8b4d8.jpg") ```
Rather than just printing data R-style, you can nicely format it into a table using kable (part of knitr). I also style mine using kableExtra, which makes it look nice and gives you extra options. By default tables fill the full width, you can override this using e.g. kable_styling(full.width = FALSE, position = “left”). When making a latex document, use kable(table, booktabs = T, “latex”) to get a (reproducible) latex-style table.
Here’s how to use python and bash. Thanks to the package reticulate, you can even share objects between your R and Python chunks. Exclude reticulate (knitr::opts_chunk$set(python.reticulate=FALSE) if you prefer to keep your languages separate.
### Mix it up with python ```{python} a='Wow python' print(a.split()[0]) ``` What a wild ride. ### or bash ```{bash, echo=TRUE} ls | head ``` Oh look, there's our output, ready to share.
Finally, if you hate GUIs – and you know I do – you can ditch the interactive notebook part and just generate documents from R Markdown files like this:
rmarkdown::render("BlogExample.Rmd")