AWS, Cloudflare, and Google have been hit with record-breaking DDoS attacks that exploit a zero day (allocated CVE-2023-44487) in the HTTP/2 protocol that is used by every modern web server – with the approach to the flurry of (largely thwarted) attacks being dubbed HTTP/2 Rapid Reset.
The vulnerability can also be exploited to execute DDoS attacks on the ubiquitous NGINX open source web server as well as the paid-for NGINX Plus, and related products that implement the server-side portion of the HTTP/2 specification, NGINX’s owner F5 said today, urging action.
“To protect your systems from this attack, we’re recommending an immediate update to your NGINX configuration” F5 wrote, adding in a short blog that it will be issuing its own patch on October 11.
From Apache Tomcat to grpc meanwhile, open source communities have been assessing exposure and pushing fixes in recent days. (The issue was identified and disclosure coordinated by AWS, Cloudflare, and Google.)
Their blogs are here:
AWS
Cloudflare
HTTP/2 Rapid Reset DDoS
Google said it had seen attacks peak at 398 million requests per second. AWS has seen attacks of 155 RPS and Cloudflare of 201 RPS.
The attacks began in late August and ran through September with decreasing success. No attribution for them was given.
Cloudflare said: “Concerning is the fact that the attacker was able to generate such an attack with a botnet of merely 20,000 machines.”
The security and CDN firm added in a detailed blog: “There are botnets today that are made up of hundreds of thousands or millions of machines. Given that the entire web typically sees only between 1–3 billion requests per second, it's not inconceivable that using this method could focus an entire web’s worth of requests on a small number of targets.”
“Most malicious requests were never forwarded to the origin servers. However, the sheer size of these attacks did cause some impact. First, as the rate of incoming requests reached peaks never seen before, we had reports of increased levels of 502 errors seen by clients.
“This happened on our most impacted data centers as they were struggling to process all the requests. While our network is meant to deal with large attacks, this particular vulnerability exposed a weakness in our infrastructure,” said Cloudflare in its blog, adding that “in addition to a direct fix, we have implemented several improvements to the server's HTTP/2 frame processing and request dispatch code.”
Google added: “The HTTP/2 protocol allows clients to indicate to the server that a previous stream should be canceled by sending a RST_STREAM frame… This attack is called Rapid Reset because it relies on the ability for an endpoint to send a RST_STREAM frame immediately after sending a request frame, which makes the other endpoint start working and then rapidly resets the request.
“The HTTP/2 Rapid Reset attack built on this capability is simple: The client opens a large number of streams at once as in the standard HTTP/2 attack, but rather than waiting for a response to each request stream from the server or proxy, the client cancels each request immediately. The ability to reset streams immediately allows each connection to have an indefinite number of requests in flight. By explicitly canceling the requests, the attacker never exceeds the limit on the number of concurrent open streams. The number of in-flight requests is no longer dependent on the round-trip time (RTT), but only on the available network bandwidth,” it explained in its blog.
“Any enterprise or individual that is serving an HTTP-based workload to the Internet may be at risk from this attack… Web applications, services, and APIs on a server or proxy able to communicate using the HTTP/2 protocol could be vulnerable. Organizations should verify that any servers they run that support HTTP/2 are not vulnerable, or apply vendor patches for CVE-2023-44487 to limit impact from this attack vector” Google added.
