Is there any way to get a VHS tape of anywhere between 2 and 4 hours down to a reasonable size while still keeping it watchable?
It will be difficult. Maybe I do not know ome of the more advanced encoding tricks, but it basically comes down to this: VHS is analog, so it has noise (it may be picture noise ("snow") or timebase errors), noise, being random, does not compress well. You should buy a TimeBase Corrector, it should reduce the timebase errors. Then you can filter out the picture noise somehow. to make it more compressible.
By the way, VHS has 480i (NTSC) or 576i (PAL). What is reduced, is the horizontal resolution, that is, while the maximum resolution for PAL is roughly 720x576, VHS has something like 300x576.
Also, VHS is interlaced. If you intend to watch the file on a CRT TV, encode it as interlaced (x264 supports that), otherwise you will have to deinterlace first.