|
|
|||||||
 |
|
|
Thread Tools | Display Modes |
|
#1
03-23-2025, 08:16 AM
|
|||
|
New research resource: EQ Archives Search Engine V2 now in beta
Hiya,
Before Christmas I resolved to teach myself a bit more Python so I could start dabbling in AI/ML. To that end I decided to start a new version of the search engine for the EQ Archives. Built with a Python backend and a heavily modified Elastic Search-UI front-end in React, the new engine has already indexed about a million documents and should be done a full index by the end of the month. I have four RTX 3090s here generating embeddings for the first pass, and further LLM enrichment will take place in the second pass, though I've already done a few domains as a demo. This engine allows you to search the EQ Archives for websites, newsgroup posts and Yahoo Groups mailing list posts. It links back to the source in the Wayback Machine and in my archive. Feel free to try it out. https://search-beta.eqarchives.org/ | ||
|
|||
|
#2
03-23-2025, 08:55 AM
|
|||
|
The next step is twofold:
* Generate data sets from the archive and fine-tune models like Gemma 3 on the content to become "domain experts" in EQ, and even specific EQ domains like Tradeskills * Use the embeddings to enable RAG and agentic chatbots with reasoning models like QWQ-32b to do deep research. You give them a question and they crawl the archive themselves to give an answer, possibly leveraging other fine-tuned models as "experts". The future is very interesting. | ||
|
|
|||
|
#5
03-23-2025, 11:17 AM
|
|||
|
Looks amazing Dolalin (and as a fellow dev I can appreciate how much work you put into it)!
__________________
Loramin Frostseer, Oracle of the Tribunal <Anonymous> and Fan of the "Where To Go For XP/For Treasure?" Guides Anyone can improve the wiki! If you are new to the Blue or Green servers, you can improve the wiki to earn a "welcome package" of platinum and/or gear! Send me a forum message for details. | ||
|
|
|||
|
#6
03-23-2025, 04:43 PM
|
|||
|
I hereby vote to Sticky this thread. The search engine is the most powerful research tool out there and should always be visible for newer folks that want to ClassicQuest
__________________
Wedar - Level 60 Grandmaster (Retired)
| ||
|
|
|||
|
#7
04-20-2025, 04:24 PM
|
|||
|
The new engine has finished its initial indexing and is now fully live at https://search.eqarchives.org
Turning off the old version now frees up resources on my server that can be used for further document enrichment. | ||
|
|
|||
|
| Thread Tools | |
| Display Modes | |
|
|
Linear Mode
