The best, cheapest and fastest way is the Gatwich Express train. It’s about 15GBP and only takes 30 minutes to get to Victoria Station in London. From there you can catch the Tube to the nearest station to your hostel.

Yes they have ATM machines in the airport. Try to find one that is affiliated with your bank as they will not charge you any fees.

Try to stay away from the exchange booths as they give you about 20% less than the correct exchange plus a fee to change your money. Also whatever you do, do not take a cab, they will cost you about 50GBP ($105) for the trip into town. The Gatwich Express is a standard train (not the Tube), it only goes from the airport to the City and doesn’t stop in-between. Learn to use the Tube, get a travel pass (will save a lot of money) and the subway is the best way to get around. It’s so easy to use.

yes there are ATMs there. Don’t bother taking a bus, it will take far too long. Go by train. You can either take the fast Gatwick Express train which goes non-stop to London Victoria in about 35 minutes and costs (I think) about £12 one way; or you can take the ordinary train, again to Victoria, which is cheaper but stops a lot on the way.
From Victoria, it’s very central to most areas – I advise you to buy an A-Z street atlas so you can work out how to get about.