After scouring the internet I found out the condition is called keratosis pilaris, which is sometimes called chicken skin (I know, it sounds disgusting!)
After to-ing and fro-ing, looking at what solutions other people have found (if they have found something, it turns out it's quite difficult to treat) I settled on the Clinique Turnaround Body Smoothing Cream which quite a lot of people have recommended.

I'm so excited for it to arrive and to give it a try! My arms have been red and bumpy for literally as long as I can remember so the prospect of maybe getting rid of it is amazing!
I'll give it a week when it arrives and post an update on whether there's been any improvements!