Project 1999

Go Back   Project 1999 > Green Community > Green Server Chat

Closed Thread
 
Thread Tools Display Modes
  #1  
Old 12-08-2020, 04:16 PM
Dolalin Dolalin is offline
Planar Protector

Dolalin's Avatar

Join Date: Oct 2009
Location: UK
Posts: 2,545
Default Searchable portal for the Everquest Archives

Some may be aware that I do a lot of research.

Over the course of the past two years I've collected over 4 million files while dumping Everquest-related websites, deprecated Yahoo Groups mailing lists, and more.

Making these easily searchable has been a constant puzzle. But I've now been able to leverage Azure Cognitive Services and Azure Cognitive Search to create a searchable portal to the eq archives.

Presenting, the Everquest Archives search portal:

https://eqarchives.azurewebsites.net/

So far I have been able to index 335,000 of the 4 million files, including all the Yahoo Groups mailing lists. They are context-searchable and metadata fields will give you links to the original Wayback Machine archive link.

Indexing the archives is proving expensive, so I will be soliciting donations at a future time to continue the process. For now, you can enjoy the 335,000 indexed documents, which include the Sony boards, EQ Stratics, Casters Realm, and a few others (alphabetically, the contents of /websites/ in the Git archive up to the letter E).

Let me know what you think!
Attached Images
File Type: jpg eq-archives-portal-front.jpg (17.7 KB, 4 views)
File Type: jpg eq-archives-portal-results.jpg (21.2 KB, 4 views)
File Type: jpg eq-archives-portal-result-transcript.jpg (21.3 KB, 5 views)
File Type: jpg eq-archives-portal-result-metadata.jpg (21.8 KB, 4 views)
  #2  
Old 12-08-2020, 04:23 PM
loramin loramin is offline
Planar Protector

loramin's Avatar

Join Date: Jul 2013
Posts: 9,340
Default

[You must be logged in to view images. Log in or Register.]
__________________

Loramin Frostseer, Oracle of the Tribunal <Anonymous> and Fan of the "Where To Go For XP/For Treasure?" Guides
Anyone can improve the wiki! If you are new to the Blue server, you can improve the wiki to earn a "welcome package" of up to 2k+ platinum! Message me for details.
  #3  
Old 12-08-2020, 04:28 PM
Danth Danth is offline
Planar Protector


Join Date: Oct 2009
Posts: 3,407
Default

Funny enough I tried to PM you a couple weeks ago as to whether you could search for some specific things, except I couldn't because you seem to have forum PMs turned off. What a nice looking resource you've built. There are a few items I intend to search for as time and motivation permit.

Danth
  #4  
Old 12-08-2020, 04:35 PM
Dolalin Dolalin is offline
Planar Protector

Dolalin's Avatar

Join Date: Oct 2009
Location: UK
Posts: 2,545
Default

I noticed there's a bug on mobile where if you search for something and press the enter button instead of the magnifying glass, your result gets reset to * on the results page. Really annoying, I will try to fix it when able.

For now, if you're on mobile, press the magnifying glass to start your search.
Last edited by Dolalin; 12-08-2020 at 04:41 PM..
  #5  
Old 12-08-2020, 04:35 PM
mcoy mcoy is offline
Planar Protector


Join Date: Mar 2017
Posts: 2,182
Default

This is awesome! Any chance you have Graffe's on your list of sites to index/archive?

-Mcoy
  #6  
Old 12-08-2020, 04:38 PM
Dolalin Dolalin is offline
Planar Protector

Dolalin's Avatar

Join Date: Oct 2009
Location: UK
Posts: 2,545
Default

Quote:
Originally Posted by mcoy [You must be logged in to view images. Log in or Register.]
This is awesome! Any chance you have Graffe's on your list of sites to index/archive?

-Mcoy
It's in the Git archive but I haven't gotten that far yet with indexing (G is after E, hehe).

It is quite expensive to index this. I need to firm up the pricing from Azure then I will ask for donations to continue the process.
  #7  
Old 12-08-2020, 04:43 PM
Izmael Izmael is offline
Planar Protector

Izmael's Avatar

Join Date: Jul 2015
Posts: 2,289
Default

Great initiative.

I can provide mirroring of this if needed, just let me know. Never know what will happen to the main site in a few years.
  #8  
Old 12-08-2020, 04:55 PM
Dolalin Dolalin is offline
Planar Protector

Dolalin's Avatar

Join Date: Oct 2009
Location: UK
Posts: 2,545
Default

The hosting is pretty easy, the site is just a tiny .netcoreapp, but it's backed by an Azure Search Service which is.... harder to mirror. I'm actually not sure if the data can be exported. [You must be logged in to view images. Log in or Register.]
Attached Images
File Type: jpg cognitive-search-stats.jpg (24.0 KB, 3 views)
  #9  
Old 12-08-2020, 04:59 PM
Dildy Dildy is offline
Sarnak


Join Date: Aug 2014
Posts: 241
Default

HNGHHHH great work! Can't wait to dive in.

Artelius
  #10  
Old 12-08-2020, 05:08 PM
Jibartik Jibartik is offline
Planar Protector


Join Date: May 2015
Posts: 22,216
Default

Quote:
Originally Posted by loramin [You must be logged in to view images. Log in or Register.]
[You must be logged in to view images. Log in or Register.]
Closed Thread

Thread Tools
Display Modes

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off

Forum Jump


All times are GMT -4. The time now is 06:11 AM.


Everquest is a registered trademark of Daybreak Game Company LLC.
Project 1999 is not associated or affiliated in any way with Daybreak Game Company LLC.
Powered by vBulletin®
Copyright ©2000 - 2024, Jelsoft Enterprises Ltd.