The machine Peter was in was found broken into many different parts and buried underground millions of years ago. The First People were rumored to have built this machine and everybody assumed that The First People had lived before the Human race came into existence.
However, that had troubled Walter. So while he was locked away in prison for years, he had time to think on The First People. It was not until he was given the information that the wormhole, which had been closed in Central Park thanks to Amber, led back nearly 200 million years ago that he finally put all the pieces together.
The First People did not live millions of years ago, but rather lived sometime in the near future. In fact, The First People were most likely a subgroup of whomever existed in 2026. Not only that, but he realized he was the one who built the machine, he was the one who sent it back in time.
But he couldn't possibly just NOT sent it back in time. That would create a paradox -- if he didn't send the machine back, then none of the events that led him to decide not to send the machine back would have happed and thus, he would have sent the machine back -- and it simply is impossible to create such a paradox. He had already sent it back, it had already been used by Peter, and he would have to send it back again. So the machine is a constant, it cannot simply be removed from the timeline.
The choice of what Peter could DO with the machine is not a constant though. It's a variable. He could destroy our universe, he could destroy the Red universe, or he could bring them together in some fashion -- which is what he decided to do.
His choice of what to do with the machine would not cause a paradox, but it would spin itself off into another outcome for the future.
I hope that helps.