The universe moves causally from one point to the other, and changes states at is goes from one to another. That was the case long before humans were around to invent a concept of time. We may have invented time as a "concept" but we use it to label a real property of the universe.
As for time travel, I don't think physics has rendered travel backwards in time completely impossible (there was an interesting SciAm article on this a number of years ago). Travel forward is certainly possible. The easiest way to resolve paradoxes from traveling backwards is to have the timeline split at the point of any possible paradoxes, so that two separate universes exist (the original and the altered one).
By the way - just going back in time to the same universe you left from is altering the past, and also appears to me to violate some laws of physics, so unless you do something like split the timeline, you've got a big problem.
I read a theory once in a science journal that every time the universe is confronted with a choice (they were talking at a quantum level I think) there is a splitting of the timeline into two separate universes - one in which each choice is made. According to this theory, the timelines merge almost immediately if the choice is minor, take long to do so for more major changes, and if the change is significant enough they never merge. You could work with something like that.
But it seems to me that if you really want to make sure you don't run afoul of paradoxes at the instant the person travels back in time, you completely sever the timelines into two universes.
Even then you've got the problem of adding matter to the universe (remember that the laws of physics, and conservation of mass and energy). I suppose it is open to debate whether or not the universe remains a "closed" system if time travel is feasible. If not, then perhaps those laws don't apply.
Thanks for explaining this.
But, of course, this just shows how complicated it is. If I split the timeline, I will have multiple versions of the same characters wandering around in space and time, won't I? How on earth would I structure such a book? I already have at least four different time travellers in the plot!