So I wrote an EQ Server population parser... here's some data !

[xious]

Hey there, just wanted to say that I talked with Rogean about making the database public and while character names are left out, the data is otherwise now available here: http://p1999pop.dmsimard.com/

The next feature I'm probably going to implement is being able to click elements, like for instance:
- If you click on a class (bard), you will be able to see information about what levels they are, the guilds and zones they are in.
- If you click on a zone (karnor), you will be able to see information about what classes, levels and guilds are in the zone
- (etc)

Let me know if you have any ideas or suggestions [You must be logged in to view images. Log in or Register.]

[falkun]

Is this compiled by you (using the client) or did someone (Rogean?) parse the actual server? What were the parameters of the server data parse?

[xious]

Originally Posted by falkun [You must be logged in to view images. Log in or Register.]

Is this compiled by you (using the client) or did someone (Rogean?) parse the actual server? What were the parameters of the server data parse?

This is done client side - Rogean has access to far superior tools to see what's going on on the server but he obviously cannot open these to the public [You must be logged in to view images. Log in or Register.]

How it works essentially is that I've made a series of macros, for instance:
/who all cleric 1 10
/who all cleric 11 20
(etc)

The output of these are logged to the eqlog.txt file and then parsed and inserted/sorted into a database.
If the parser sees a newer record for a player, it updates the record in the database, it otherwise ignores it.

This allows me to eventually, as I run these macros several times a week, to catch every character at least once and end up having pretty accurate data.

[Vykenos]

This data set will also become incorrect and bloated with old data as well wont it? Such as people leaving and not logging on there characters, characters no longer being used or deleted, etc.

It might be good to start a new tally ever month or so? Or maybe dating and purging entries that are not seen again over a certain time so the information is kept fresh to represent the server?

[xious]

Sure, I guess I could have records expire after a period of time that I haven't "seen" them.

Should be fairly easy to do, too. I'll implement that, great idea [You must be logged in to view images. Log in or Register.]

[Lyra]

Originally Posted by xious [You must be logged in to view images. Log in or Register.]

he obviously cannot open these to the public

Not obvious to me - Although it's his server to do as he pleases.

Originally Posted by xious [You must be logged in to view images. Log in or Register.]

character names are left out

This information is available, including character name, to anyone playing the game. You have been so kind as to gather data in one place. Not sure why the name should be considered private information. - But again...your thing.

[xious]

Are character names relevant though ? I don't want to create a tool so people can brag/rant/flame/troll, etc.

[Thana8088]

When I try to look at a population, I get this:

Originally Posted by Error

Internal Server Error
The server encountered an internal error or misconfiguration and was unable to complete your request.

Please contact the server administrator, root@localhost and inform them of the time the error occurred, and anything you might have done that may have caused the error.

More information about this error may be available in the server error log.


--------------------------------------------------------------------------------

Could it be something on my side? Our IT people don't let us do anything!

[xious]

Nope, that was indeed a problem - fixed now though.

[August]

How do you prevent contamination of data?

Say you do a parse and I am playing as a level 10 cleric and then the next day I am level 11. I am now in 2 buckets and my character is counted twice.