Answer by Pedro Ribeiro for Why do objects following a path have a random drastic rotation?

Fixed it. On the curve properties menu (Object data properties), Shape > Twist method, changing it to Z-Up instead of Minimum fixed it for me, just tweak that or the Smooth option too and see if it works for you.