Server Emergency Patch: Dupe Fix

[Rogean]

On Monday, November 8th, the server alerted me to a player holding 21 Million platinum. Upon confronting the owner, he refused to acknowledge his guilt and work with staff to reveal the method being used. All accounts sharing his IP, and any other IP's having been used and any accounts using those IP's, were banned (about 5-6 total).

An extensive investigation was started, digging through all sorts of logs, which lead us to the certain mechanics being used. After thorough testing and exploiting of those mechanics, we discovered the method being used. 90 Minutes later, on November 10th at 1 AM, the server was brought down to patch this hole.

I just wanted to make this post to let the community know what happened, and how serious we take this. The integrity of our game economy is an important part of what makes this server a fun place to play on.

This occurance was caught early by our detection flags and was able to be mostly contained. Some high value items were bought, and we will be going through and reversing those trades (You probably know who you are if you received platinum from this person, so be ready for that).

A huge thanks to Uthgaard for the extensive log digging and testing he did, and to Haynar for helping me with checking over the code.

[Dac321]

Good job. Thanks for keeping the game clean.

[audacity]

woot. Good job P99 crew :D

[rainingvodka]

thank you very much rogean for busting this perp.

(and just as a side note, I think u have your dates a little off tho)

[Morlaeth]

Zomg you fixed stuff in the FUTURE?!?!?!

But seriously, thanks for keeping everything intact. You guys rule.

[Lanvaren]

Nice find.


.. I wonder if we'll see a CoF for sale (again) soon? :p

[Rogean]

lol fixed dates

[nalkin]

lol why would he hold such a conspicuous amount of money? Why not just hold like 200k or something half-way believable? There must have been some logic behind this... these are the details I crave Rogean.

[RKromwell]

Good job guys.

[purist]

50 bajillion platz

[Dumesh Uhl'Belk]

Woot! thanks Rog

[Kich]

Hopefully 21 million plat wasn't the red flag haha. And yeah, I don't get the mentality. If you're someone capable of doing something like that--typically you (should) also know how you could be caught. Not to mention, nothing in the game goes for anywhere near that much plat..

[Zithax]

Great job guys.

SUP poorSPORT?

[Stibe]

Right on guys. +100 internets for you all.

[Fourthmeal]

Rogean really thought outside the bun on this one!

[Namae]

/thumbsup for the quick squashing!

[warrioman]

So how was it done now that it has been fixed - I'm interested to see to what lengths people go to hax.

[Uthgaard]

It was more obscure than difficult. The high profile shopping spree the fool went on should be a testament to the guy's intelligence. Most bugs are found by accident.

[DekThai]

The guy got stupid greedy, he bought hammerhead Helm for 55K PP from my guildie. LOL

[Lill-Leif]

I think the big question here is: Was he encumbered??

[Darklake]

lol why would he hold such a conspicuous amount of money? Why not just hold like 200k or something half-way believable? There must have been some logic behind this... these are the details I crave Rogean.

All the gear with no idea. He could have kept a low profile and just tick along quietly, drawing no attention to himself, kitting out slowly with high end stuff. Either a kid with no clue or someone who fancied themselves as a "plat for cash" business.

[Snigel]

Intriguing, how was it done?

[Omnimorph]

The details for these things are never posted by the devs so no point asking.

I can only imagine it was a type of thing where the user feared it might get patched one day so thought he'd milk it for all it's worth now and sit on the plat...

[Madigan]

Awesome catch and follow through! This level of attention to the server is sure to help retain players and hopefully bring in others via word of mouth.

[ukaking]

Nice work fellas.

[Deathrydar]

I think the big question here is: Was he encumbered??

I would like to know the answer to this questions as well!! :D

But seriously folks, kudos to the Project 1999 team for always keeping a watchful eye out for the good of the server!

[Lazortag]

lol why would he hold such a conspicuous amount of money? Why not just hold like 200k or something half-way believable? There must have been some logic behind this... these are the details I crave Rogean.

I'm a little concerned that they admitted to how much plat they found him holding. Now in the future when I dupe I'll have to pull an Obama and "spread the wealth around" if you catch my drift ;)

In all seriousness I'm very glad to hear about how efficiently this was dealt with. I just hope I never traded with him!

[Ploppy]

I think the big question here is: Was he encumbered??

lol

[Lickum]

Rogean really thought outside the bun on this one!

You should try thinking outside the bun for some new material.

[skorge]

you just got served Fourthmeal

[Rogean]

you just got served Fourthmeal

Why doesn't anyone serve me a fourthmeal :(

[Harkor]

Not sure if it is still like this, but after the server restart the raft in Halas was glitched.....2 stacked on top of one another. Just thought I'd let ya know!

[krforrester]

Was the character's name Obamazz? I think the plat was to help fund the new controversial Cleric Complete Heal Plan.

[Trimm]

Aww, and here I was hoping the 875k offer I got on my RBB was legit. Shucks.

[Lanvaren]

Not sure if it is still like this, but after the server restart the raft in Halas was glitched.....2 stacked on top of one another. Just thought I'd let ya know!
One of those rafts just might be a player's corpse.

I had a wild night in Halas...

[Fawqueue]

You know, growing up my father was always fond of embarrassing me when I had done something wrong in order to ensure I wouldn't make the mistake a second time. And it was effective, knowing that everyone knew and gave me a hard time was a good deterrent. So in the spirit of that, any chance we could find out who this guy was? That way if any of us were friends with him, we can harrass and ridicule him to no end. Plus if he finds a way back onto the server again under another new account, maybe he'll know better this time. It's a win!

[Shannacore]

Why doesn't anyone serve me a fourthmeal :(

alfredo, spoon fed.

[Lyrik]

You know, growing up my father was always fond of embarrassing me when I had done something wrong in order to ensure I wouldn't make the mistake a second time. And it was effective, knowing that everyone knew and gave me a hard time was a good deterrent. So in the spirit of that, any chance we could find out who this guy was? That way if any of us were friends with him, we can harrass and ridicule him to no end. Plus if he finds a way back onto the server again under another new account, maybe he'll know better this time. It's a win!

If I may, I'll translate this:

In case that person makes it back I'd like to ask them how they did it.

[hedbonker]

What a genius...

Reminds me of Superman III where Richard Prior exploits a rounding error at the bank and the CEO is having a meeting about the issue and how they will never catch the guy unless he is a complete moron and shows off the wealth - And at that moment Richard Prior pulls into the parking lot in a new Ferrari burning rubber all the way...

[guineapig]

I'm a little concerned that they admitted to how much plat they found him holding. Now in the future when I dupe I'll have to pull an Obama and "spread the wealth around" if you catch my drift ;)

In all seriousness I'm very glad to hear about how efficiently this was dealt with. I just hope I never traded with him!


This actually brings up a very good point. The person was caught because he kept the majority of his plat on 1 character on 1 account. However if a person manages to spread it out over multiple characters on multiple accounts this might be avoidable.

If there isn't already one in place, I would humbly suggest some sort of process where by you can detect the total net platinum a player has based on adding up the total platinum a player has on all characters and accounts connected to that player.

Come to think of it, stackable vendor sold stuff, such as valuable gems and high value alchemy components (basically anything stackable that a vendor sells unlimited amounts for for 50+ plat a piece) are a way too easy way for people to launder in game money and make it basically vanish from their inventory until a time when they are ready to use it. So tracking the sale of unusually large quantities of such items might also be prudent.

[Rogean]

Many good suggestions. Some are already in place, some will be put in. Backend logging and flags are things I have been working on for quite some time now.

[Villide]

Many good suggestions. Some are already in place, some will be put in. Backend logging and flags are things I have been working on for quite some time now.
Thanks Rogean, the efforts of you and your crew are greatly appreciated!

[Uthgaard]

I won't go into detail about all of the methods I personally use to keep tabs on things top of the server detection, but yeah. We've got you covered.

[YendorLootmonkey]

I always figured the server kept track of roughly how much currency was in the game at any given time. Considering currency only enters the game via quests, vendor sales, or mob drops, and only leaves the game via destroying coins, vendor purchases, or quest turn-ins (trading stuff for plat doesn't impact the total currency in game, it just shifts the distribution of it across characters, so you wouldn't need to track this)... it should be easy to track this "rate of net currency increase" in the game and what the average rate is.

Have red flags go off when the net currency increase jumps way beyond that rate within a given time period. That would give you a pretty good indicator that currency entered the game via a dupe, because it seemingly came out of nowhere. Or if particular mobs were dropping an abnormal amount of plat after a patch.

[rachel]

I knew that guy Sport/bonssum? that was offering 600k+ for the rbb/cof.
I thought to myself how the hell did he get that much plat?
Than I thought maybe his guild disbanded and he sold his guild bank.

It sucks because it's hard for me to make 1k a day.

Thank you all for catching him! It sure seemed fishy to me.

[Dantes]

lol why would he hold such a conspicuous amount of money? Why not just hold like 200k or something half-way believable? There must have been some logic behind this... these are the details I crave Rogean.

This was my first thought too. 21 million plat? Who the fuck will need 21 million plat EVER? And he denied it? I wonder what his story was? "I earned it. All 21 million, check the logs, I have 50,000 hours played."

Maybe he messed up some little mundane detail like he put a decimal point in the wrong position.

[Fawqueue]

If I may, I'll translate this:

In case that person makes it back I'd like to ask them how they did it.

I think you'd have to be an idiot to read this post, KNOW that they have systems in place to identify these problems, KNOW that they've thus fixed it, and still think "Hey, maybe I'll try this out myself!"

I just wanted to know who it was, so that if I did know him...I could tease him mercilessly because he was a dipshit =P

[Engraverwilliam]

once i figured out a way to solo multiple drovargs in karnors by using a pathing issue on live. I gained 4 levels doing this. After which I was feeling so guilty I remained paranoid for the longest time tht some GM would show up while I was out hunting and say "remember that time in Karnors?" cheating only cheats yourself.

[yraapt]

Glad to hear the dupers got banned!

I have zero knowledge of the emu code, but it seems like there could be a way to track when players repeatedly do the same steps XYZ over and over again. Seems like this would trigger for both dupers and auto macro stuff. Although I'm guessing the devs would have already done this if possible... that or they are already doing it and we just don't know. lol

[ryuut1]

wow, 21 mill plat. that's ...too much.

[Lyrik]

I think you'd have to be an idiot to read this post, KNOW that they have systems in place to identify these problems, KNOW that they've thus fixed it, and still think "Hey, maybe I'll try this out myself!"

I just wanted to know who it was, so that if I did know him...I could tease him mercilessly because he was a dipshit =P

It was a joke :(

[hedbonker]

I guess I just fail at the whole idea of cheating. Why play this if you are going to cheat? You make plat and get items that are commensurate with your level. It's a journey to be enjoyed. So when you actually bank that first 1K there is a sense of accomplishment associated with it.

[Rogean]

I guess I just fail at the whole idea of cheating. Why play this if you are going to cheat? You make plat and get items that are commensurate with your level. It's a journey to be enjoyed. So when you actually bank that first 1K there is a sense of accomplishment associated with it.

The majority of people would agree, but then a lot of them, when it comes to the opportunity (through the accidental discovery of a bug or whatnot), do not take the honest approach of reporting it. Even those that do, most do so out of fear of being caught and banned.

[Zithax]

Btw this guy offered me 800k for my rbb, 600k for my gaunts of fiery might. i was like uhhh let me think about it? (no?) Then I auctioned for spined dragon claws. He was like he has them for 20k, but instead offers 1.5mil for the rbb and gaunts.

LOL YEAH NO

[Rogean]

lol @ divide by zero tag.

THAT WOULD BE A SEGFAULT.

[hedbonker]

Well, I guess greed comes into it at some point and people get blinded by that...

[dvs198]

Nice job guys good to know you'r looking out for us =)

[Brund the Decrepit]

Awesome catch and foot work by Rog and Uth!
Bad boys... bad boys.........

[vitalious]

I'd like to thank the exploiter and the gms. Thank you for finding this retard, but more importantly, thank you for bringing down the server and brining it up right when I got home from work. Allowing me to pick and choose my group and camp and grind out of 40. THANKS :)

[Mimn]

Thanks Rogean, Uthgaard, and crew for taking in game monetary policy seriously. On live, seemingly the only GM I ever knew who personally understood how integral this is to maintaining a healthy play environment was Uini. I would assert that runaway deflation 'killed' EQ more thoroughly than any item, code, or class balance issues.


ETA: Did it by chance have anything to do with the Fear portal zone-in or has that duping bug already been addressed?

[Hasbinbad]

Aww, and here I was hoping the 875k offer I got on my RBB was legit. Shucks.
lol, he offered me 1.1mil, a fbss, and a yak..
first person i asked advice from was rogean XD

[Mountaineer]

Whew, good thing I didnt get flagged for having 19 million plat!

[Rogean]

afayk.

[JayDee]

The guy who did it is from vztz and they have alrdy duped a ton of plat here (according to them).

I remember a gm from vztz saying that EQ has too many dupe methods to count. Makes me feel sorry for legit players like Alawen who grind platinum all day.

[Cyrius]

This was my first thought too. 21 million plat? Who the fuck will need 21 million plat EVER? And he denied it? I wonder what his story was? "I earned it. All 21 million, check the logs, I have 50,000 hours played."

Maybe he messed up some little mundane detail like he put a decimal point in the wrong position.

Would take a mage only 21000 hours in OOT. Thats only 875 days or ~2,4 years of online time.

[Taluvill]

Would take a mage only 21000 hours in OOT. Thats only 875 days or ~2,4 years of online time.

And the server has been up for like a year.

[cadiz]

Thanks Rogean, the efforts of you and your crew are greatly appreciated!

+1 thanks for taking a proactive approach to keeping the server clean and fair, it's much appreciated.

[Zarcath]

Thanks Rogean, Uthgaard, and crew for taking in game monetary policy seriously. On live, seemingly the only GM I ever knew who personally understood how integral this is to maintaining a healthy play environment was Uini. I would assert that runaway deflation 'killed' EQ more thoroughly than any item, code, or class balance issues.


ETA: Did it by chance have anything to do with the Fear portal zone-in or has that duping bug already been addressed?

Well Uini had some kind of business economics degree and the economy for EQ wasn't designed to last for more than 2 years.

The economy thing came up at a GDC roundtable a few years ago that was headed by some SoE peeps and they talked more about it, but it got derailed into some stupid World War 2 Online economic shit and some other terrible game that came out around that time. etcetc.

[Uthgaard]

A game is defined very heavily by its economy. Time is money is more than just an expression, it's an intermediate currency that represents the amount of time it takes a player to generate value. The value of items is influenced by its quality in comparison to other items, and the numeric value of the price tag is going to be directly proportional to the amount of time it takes to acquire the item, and inversely proportional to the amount of money that a player of the average level on the server can be earned in the same amount of time. Then supply vs demand brings it up or down accordingly.

So when a player finds a method to introduce plat to the economy, eventually the means of acquiring plat by playing the game begin to lose value (for everyone) - and logically following, so does playing the game, as content is rendered useless - and players entering the game find themselves at a huge disadvantage.

So, it's important for the health of the game to prevent situations like this when they occur, identify them rapidly, and reverse the damage done. But don't think that we depend solely on some server red flag to identify plat entering the game, or that it's set to some arbitrarily high number because the guy got to 21 million in one day, or that we wouldn't have known if he hadn't been flagged. There are a lot of methods - both active and passive - that are taken to keep on top of things.

[Snigel]

Sounds good!
What are the developer thoughts on reducing the money supply, like the casinos did on Live?

On live it was used as a reactive method, have you thought about a preventive method?