Looking into you bookreading analogy, convinces me even more that the "old" blue and red timeline is no longer available.
Think about it. At the end of season 3 Peter left the blue universe we (and he) knew. His been living in this alternate blue universe for quite some time now.
He is no longer the Peter that left. (new emotions, feelings, thoughts, knowledge etc). So, if he were to go back to our blue universe, he himself, would make new decisions from then on, resulting in yet another blue universe.
You know what I mean? He stops reading the book at the end of the 3rd season, and when goes back, he doesn't start reading in the same book he was reading. He would start reading yet another book, starting on the same page where he left of. So right up till the moment he left, this new blue universe would be the same as the old one, but from then on, it would be another one
Which is why I think that if timetravel were possible, going to the past would always result in never beeing able to go back where you came from...
As for the observers, I'm still not convinced that they are more then mere drones. Has anybody read Insomnia (Stephen King). Sometimes, the observers remind me of the "little white doctors". They have a task, and the abilities to perform that task, but nothing more. I'm still thinking their being controlled or at least supervised by a higher power. Someone (or more) higher on the ladder, you know
Same remark for the machine. I can't shake the feeling that we're not looking at the bigger picture here. We're beeing mislead by the writers. It is true what you said about season 3. All the manoevering of the observers to get Pater in the machine....
I'm rewatching everything for the third time, and still it is beyond my reach.