Monthly Archives: February 2023

The Most ReLU-iable Activation Function?

The Rectified Linear Unit (ReLU) activation function was first used in 1975, but its use exploded when it was used by Nair & Hinton in their 2010 paper on Restricted Boltzmann Machines. ReLU and its derivative are fast to compute, and it has dominated deep neural networks for years. The main problem with the activation function is the so-called dead ReLU problem, where significant negative input to a neuron can cause its gradient to always be zero. To rectify this (har har), modified versions have proposed, including leaky ReLU, GeLU and SiLU, wherein the gradient for x < 0 is not always zero.

A 2020 paper by Naizat et al., which builds upon ideas set out in a 2014 Google Brain blog post seeks to explain why ReLU and its variants seem to be better in general for classification problems than sigmoidal functions such as tanh and sigmoid.

Continue reading

SnackGPT

One of the most treasured group meeting institutions in OPIG is snackage. Each week, one group member is asked to bring in treats for our sometimes lengthy 16:30 meeting. However, as would be the case for any group of 25 or so people, there are various dietary requirements and preferences that can make this snack-quisition a somewhat tricky process: from gluten allergies to a mild dislike of cucumber, they vary in importance, but nevertheless are all to be taken into account when the pre-meeting supermarket sweep is carried out.

So, the question on every researcher’s mind: can ChatGPT help me? Turns out, yes: I gave it a list of the group’s dietary requirements and preferences, and it gave me a handy little list of snacks I might be able to bring along…

When pushed further, it even provided me with an itemised list of the ingredients required. During this process it seemed to forget a couple of the allergies I’d mentioned earlier, but they were easy to spot; almost more worryingly, it suggested I get a beetroot and mint hummus (!) for the veggie platter:

I don’t know if I’ll actually be using many of these suggestions—judging by the chats I’ve had about the above list, I think bringing in a platter of veggies as the group meeting snack may get me physically removed from the premises—but ChatGPT has once again proven itself a handy tool for saving a few minutes of thinking!

SUMO wrestling with developability

When engineering antibodies into effective biotherapeutics, ideally, factors such as affinity, specificity, chemical stability and solubility should all be optimised. In practice, we know that it’s often not feasible to co-optimise all of these, and so compromises are made, but identifying these developability issues early on in the antibody drug discovery process could save costs and reduce attrition rates. For example, we could avoid choosing a candidate that expresses poorly, which would make it expensive to manufacture as a drug, or one with a high risk of aggregation that would drive unwanted immunogenicity.

On this theme, I was interested to read recently a paper by the Computational Chemistry & Biologics group at Merck (Evers et al., 2022 https://www.biorxiv.org/content/10.1101/2022.11.19.517175v1). They have developed a pipeline called SUMO (In Silico Sequence Assessment Using Multiple Optimization Parameters), that brings together publicly-available software for in silico developability assessment and creates an overall developability profile as a starting point for antibody or VHH optimisation.

Read more: SUMO wrestling with developability

For each sequence assessed, they report factors such as sequence liabilities (residues liable to chemical modifications that can alter properties such as binding affinity or aggregation propensity), surface hydrophobicity, sequence identity compared to most similar human germline and predicted immunogenicity (based on MHC-II binding). Also provided are an annotated sequence viewer and 3D visualisation of calculated properties. Profiles are annotated with a red-yellow-green colour-coding system to indicate which sequences have favourable properties.

Overall, this approach is a useful way to discriminate between candidates and steer away from those with major developability issues prior to the optimisation stage. Given that the thresholds for their colour-coding system are based on data from marketed therapeutic antibodies, and that the software used has primarily been designed for use on antibody datasets, I would be interested to see whether the particular descriptors chosen for SUMO translate well to VHHs, or whether there are other properties that are stronger indicators of nanobody developability.

Datamining Wikipedia and writing JS with ChatGTP just to swap the colours on university logos…

I am not sure the University of Oxford logo works in the gold from the University of Otago…

A few months back I moved from the Oxford BRC to OPIG, both within the university of Oxford, but like many in academia I have moved across a few universities. As this is my first post here I wanted to do something neat: a JS tool that swapped colours in university logos!
It was a rather laborious task requiring a lot of coding, but once I got it working, I ended up tripping up at the last metre. So for technical reasons, I have resorted to hosting it in my own blog (see post), but nevertheless the path towards it is worth discussing.

Continue reading

Creating a Personal Website

Personal websites are a great and increasingly important way to build your online presence. Along with professional social media pages, such as on LinkedIn and Twitter, a website can provide a boost to your career and/or job search.

This blog post is based on my recent experience creating a personal website, following guidelines from Lewis’ talk at the OPIG Retreat last year (thank you Lewis!). The method I used and will cover here, based on an HTML5 UP! template and GitHub pages, is free and fast.

Why have a personal website?

  • Improves your online presence and brand
  • Boost for your career, including by allowing potential future employers to find you
  • Share things you have accomplished or are interested in
Continue reading