A call to (nerdy) arms

[azxten]

Its pretty simple really. There is some volume of traffic directed at the server. Some percentage of this is malicious DDOS attack traffic.

Can that malicious traffic be differentiated from legitimate traffic?

If yes, then it can be filtered out. However, filtering takes CPU cycles and more complex rules typically require more cycles.

Where do we do the filtering? Options are on the server ($) or on an upstream router ($$) or security appliance ($$$). Doing it on your own server is cheap but its also a bit too late because now the server is handling the packet filtering thus overloading itself. Doing it on an upstream router is more expensive because XYZ data center has to maintain the rules for you but moves the load off the server. Doing it on a security appliance is most expensive because they cost to buy but is most effective due to being purpose built.

If no, then all of that traffic needs to hit the server and the server needs to keep up. ($$$$)

This typically means bringing in a load balancer and shifting to a multi-server architecture which typically means the server software will require adjustments to run properly since this is all custom code. ($$$$$)

I'm guessing the server is at the point where the attack is being mitigated on an upstream router/security appliance but its dynamic enough in nature that it can still cause outages until the firewall adjusts to the new sources.