Yea, unfortunately, had the same thing happen to me. Maybe the Beta version will work for you.
I tested Gavin out on two different computers (This one is older, and then I tested him out on a more recent computer) and I still crashed at the Temple district once I was about to leave. Both computers had the full install (As we don't like having to switch discs), and both were SOA only. Berelinde even tried making a patch to skip pass that part, but nothing seem to work.
Just recently I installed TOB and played on the SOA side, and he works fine. I haven't had any problems with him since. If the Beta version doesn't work, and you still have the same problem, then, yea, you need TOB.