https://tokyocheapo.com/travel/transport/which-jr-rail-pass-to-choose/You can do both. if you need a reserved sheet in days including holidays, you'd better book.
however i recommend Japan Rail Pass for visitors.
Most of the time, you can just go to the station. Some peak periods like the end/start of the year, Golden Week and Obon are a few exceptions though. Plus you don't say if you are buying regular fare tickets or using a full JR Pass. If the latter, you can make seat reservations for free.
You can check rail schedules, fares, and transfer info on Hyperdia.