To cover the back of the chair, I was going to do another tutorial, but then I thought, "Self, there is probably a tutorial already out there." So I googled it and this is what I found!
Easy Slip Cover
If you want one to cover the entire chair, check out THIS post.
I found that the small cover I made wasn't big enough because little fingers got other parts really dirty. I am haven't made mine yet. but I am going to make the 1st choice. Good luck!