Is it possible to download the wiki to have it locally?

[Goregasmic]

When you want to lull from out of LOS you're going to spam consider a fair bit every time to see if faction goes from X to threatening when it lands, that's your cue to bail out if it does.

I don't use pigparse UI but if it pulls up the wiki page every time...

[loramin]

Originally Posted by bcbrown [You must be logged in to view images. Log in or Register.]

Pretty sure it's you who's mistaken. I don't think you understand what PigParse is. It's not just a tunnelquest website, it's also a log-reading utility that does DPS parsing and mapping, and I guess loading all items the wiki lists as dropping from a mob every time you /con it.

Here's the ConHandler hitting the the WikiApi:
https://github.com/smasherprog/EqToo...Handler.cs#L46
Here's the WikiApi directly hitting the wiki:
https://github.com/smasherprog/EqToo...ikiApi.cs#L107

Sure don't see any caching happening.

If it's hitting the p99 wiki every time any player using pigparse /cons a mob, that's a lot more than "one page, even a few times". And sure, I'm aware of the difference between that and an non-rate-limited multithreaded crawler, but unless you have some special insight into wiki traffic from being an administrator, neither of us knows whether the wiki slowness is caused by crawlers or something else. In contrast, the issue pointed out by Mattson is a real source of a nontrivial amount of unnecessary traffic. So why bother responding dissmissively if you're not even going to put in the effort to understand what the issue is?

It seems like the maintainer is aware that this is... not a best practice, and I see some (unfinished) work today to begin to implement caching:
https://github.com/smasherprog/EqToo...0ae286a691d333

So PigParse is DDOSing our wiki? Awesome! [You must be logged in to view images. Log in or Register.]

Look, I googled and saw an auction site. I can't be responsible for every tool and knowing how they all work ... I just figured no dev would be so stupid (and cruel) as to effectively encourage users to DDOS the wiki!

But, to be fair, I'm still unclear how many users it has, how often they spam the "take down the wiki" button, etc. If it only results in a few requests per user per minute, that's the same as a human wiki user (and not a problem). But if it's making a whole bunch of requests per second, frequently ... the PigParse dev is a true jerk, and Rogean should block the tool from wiki access.

[bcbrown]

Originally Posted by loramin [You must be logged in to view images. Log in or Register.]

So PigParse is DDOSing our wiki? Awesome! [You must be logged in to view images. Log in or Register.]

Look, I googled and saw an auction site. I can't be responsible for every tool and knowing how they all work ... I just figured no dev would be so stupid (and cruel) as to effectively encourage users to DDOS the wiki!

But, to be fair, I'm still unclear how many users it has, how often they spam the "take down the wiki" button, etc. If it only results in a few requests per user per minute, that's the same as a human wiki user (and not a problem). But if it's making a whole bunch of requests per second, frequently ... the PigParse dev is a true jerk, and Rogean should block the tool from wiki access.

To be clear I'm not trying to start any beef with you. I like and respect you, and you do far more for this community than most other players.

I don't expect you to know every tool and how they work. But I do expect you to do better than dismissively deny the possibility that a tool you don't know about does <a thing>. Besides, I know you've seen the quality of code that other p99 enthusiasts have written - expecting amateurs to meet professional standards isn't very reasonable [You must be logged in to view images. Log in or Register.]

And for sure, none of us have visibility into why the wiki's so slow, whether this tool is a significant source of traffic, etc. But here's a simple Fermi estimate: 100 average concurrent Pigparse users who /con every 10 seconds on average vs 2000 Wiki users who view 100 pages a day on average.

Also, Rogean can't easily block the tool from wiki access since it's client-side logic, not server-side.

[loramin]

You've got me: I thought anyone coding a tool that hits the wiki would also understand how requests and servers work ... and with that understanding, wouldn't still write their code in a way that hurts the wiki.

So yeah, I'm a terrible person/poster for assuming other programmers aren't terrible people (and having to assume because I don't know every tool and how it's coded).

As for Rogean not being able to block the tool ... there are ways. Tools have patterns and make requests with those patterns. Again, I still have no idea if PigParse has even contributed, meaningfully, to the wiki issues ... but if it does, I'll personally volunteer to write code to block it from accessing the wiki ... if Rogean can't deal with it himself (I strongly suspect he can).

P.S. Also ... where did I ever rule out the idea that the tool could be responsible? Quote me, because all I see are words like "imagine" and "hope".

I truly don't understand why people on this forum need to attack others for what they imagine they've said. I've actually said plenty of things worth arguing over: pick one of them to start a fight over if you just want to argue with an Internet stranger.

[DeathsSilkyMist]

Originally Posted by loramin [You must be logged in to view images. Log in or Register.]

I truly don't understand why people on this forum need to attack others for what they imagine they've said.

You do this often. It is indeed strange as to why posters like yourself do this.

For all of the disagreements myself and Bcbrown have had, I don't see him attacking you here. He is simply asking you to listen to others, instead of simply dismissing them out of hand. You did indeed do that to Bcbrown.

[bcbrown]

DSM, showing up with nothing other than offtopic personal attacks ("You do this often.") fits within my definition of trolling. I appreciate the support but I know you can do better than this.

I don't think you're a terrible person or poster, Loramin. If you thought I was attacking you I apologize. I thought you made a mistake, and the thing about mistakes is we all make them. Let's get back on topic, shall we? [You must be logged in to view images. Log in or Register.]

Mattson, I wrote up a little tool that you (or anyone else) might try out and see if it could be useful. It's a local webserver that proxies the p99 wiki. So you can browse the wiki just like normal, but every page you visit gets cached locally. That means that if the wiki is down or slow it will fall back to the local cache.

It's only half-finished and still quite rough around the edges but if it proves to be useful I can polish it up.

Instructions
1. Install Python3 on your computer (https://www.python.org/downloads/)
2. Download the attached main.txt file somewhere on your computer - the cached wiki files will be downloaded to this location
3. Rename from main.txt to main.py
4. Open a command prompt or terminal at that location
5. type "python3 -m pip install requests" to install the necessary 'requests' library
6. type "python3 main.py"
7. open a web browser and go to "localhost:8088/Main_Page"

Current Limitations
* Search does not work. You have to navigate to the specific page by URL or links
* The wiki uses redirects (like "Skyshrine_Armor" redirects to "Skyshrine_Armor_Quests" and there's probably some corner cases that won't work correctly
* The wiki is case-sensitive ("Skyshrine_armor" is different from "Skyshrine_Armor") and there's probably some corner cases that won't work correctly
* There's no cache updates right now, so the page you see will always be frozen at the first version cached. This would be the next thing I'd work on if there's interest in this project.
* CSS styling isn't working and I don't know why, so the formatting looks all wonky.

[DeathsSilkyMist]

Originally Posted by bcbrown [You must be logged in to view images. Log in or Register.]

DSM, showing up with nothing other than offtopic personal attacks ("You do this often.") fits within my definition of trolling. I appreciate the support but I know you can do better than this.

Loramin brought it up directly (which was off topic). I wouldn't have responded otherwise. Loramin does indeed need to stick to the topic instead of trolling other posters like yourself. Loramin needs to be held to a higher standard since he is a wiki admin. He shouldn't be doing this.

[bcbrown]

Originally Posted by DeathsSilkyMist [You must be logged in to view images. Log in or Register.]

Loramin brought it up directly (which was off topic). I wouldn't have responded otherwise. Loramin does indeed need to stick to the topic instead of trolling other posters like yourself. Loramin needs to be held to a higher standard since he is a wiki admin. He shouldn't be doing this.

Loramin is not trolling anyone, but you are. You can do better.

[DeathsSilkyMist]

Originally Posted by bcbrown [You must be logged in to view images. Log in or Register.]

Loramin is not trolling anyone, but you are. You can do better.

I disagree with your assessment, and Loramin's past behavior disproves the claim Loramin doesn't troll. Please stick to the topic, as there is no need to continue this back and forth. Hopefully he will respond to you with a proper post this time.

Originally Posted by loramin [You must be logged in to view images. Log in or Register.]

You've got me: I thought anyone coding a tool that hits the wiki would also understand how requests and servers work ... and with that understanding, wouldn't still write their code in a way that hurts the wiki.

I don't think you should assume this, nor is there a reason to be snarky about it. The principle behind PigParse is simple enough to where someone could vibe-code it with AI. Not everybody frequents the forum. They may not even know that the wiki has issues. Many amateur programmers are not focused on these kinds of secondary effects. They may just be happy that they got PigParse working in some form.

Originally Posted by Swish [You must be logged in to view images. Log in or Register.]

Use the wayback machine if its being slow

http://web.archive.org

This is a good idea!