This was definitely a highlight of our trip to Iceland. The glacier is well worth your time to visit although I would caution that the drive back to it is SUPER sketchy. I wouldn’t recommend driving it in a rental car if you don’t at least have all-wheel drive and some experience navigating tricky mud puddles; also, the bus drivers will block you in, so be careful where you park.
The glacier itself is very accessible for hikers. The terminal face ramps down to the path quite nicely and you can do a little exploring with just regular shoes right at the face. We got some really neat pictures of each other in little "ice caves" and around some ice formations. Heed the warning signs though and don’t get out on the ice without a guide; you wouldn’t be the first person to never be heard from again.