Retain sun ROATATION alongside video keyframe Merged

Taking a video around a house. You can set your lighting as you're looking at the front to where it looks good. you can retain the time of day. you move to one side and set the time of day a bit different to still get sun on it in a decent position. by the time you get to the back of the house you can't keep good lighting. I would think it'd be easy to set the time of day along with the sun rotation and then as you go around the house it just works it's way to the newly set sun rotation the same way it would with time of day...game changer