I solved this issue, but i still do not really understand the logic behind it. I tried making a new page with the slider without any styling or settings and it started out on slide 2, but not centered. So i figured the problem was not the dynamic content or any margings/padding.
Did some digging on the forum and found this post: SOLVED: Nested Slider Items per page bug
Which is not exactly what i ran into, but changing the settings for the slider from loop to slide made it behave properly.