Quote:
Originally Posted by BlackBellamy
[You must be logged in to view images. Log in or Register.]
Yeah you better hope the wayback machine keeps running, because that's the only glue holding the internet together. People back in the day were like yeah the internet is forever but I ask them to find that article from 20 years ago and it's like what happened?
|
Well, even that is only a partial fix. Wayback Machine leaves out TONS of stuff.
Minor website background info: every website can create a file called robots.txt. This file gives the website a way to tell "robots" (search engines) what parts of the site they can and can't see. For instance, maybe you don't want people going in to your foo page inside your bar page directly from Google: you want them to go to the bar page first, so you block the foo page in your robots.txt.
What does this have to do with the Wayback Machine? Well, because it's a "robot" itself, it respects robots.txt, which means that if a website blocks a part of itself in that file, it won't be archived. What's worse, if it
was archived, and the site changes their robots.txt to block that part (... in 2019, for reasons having nothing to do with the Wayback Machine), the Wayback Machine will no longer show archives of that part.
This is why the entirety of the EQ Traders forums are now forever lost to time. The Wayback Machine has an archive, but won't show it until the EQ Traders robots.txt changes, and since that file is controlled by the giant message board system that EQ Traders uses ... fat chance of that ever happening.