The first time I needed to dig deeply within the understanding of the work of web server was few months ago. It was when one of the websites that I monitor got closed by the hosting company because it was consuming a huge amount of resources (specifically CPU and RAM). I fixed it after reducing the size of my web pages, removing unnecessary connections and database queries then it was working fine. However, after discussing the matter with my friend Omar Alshaker, I started to think more deeply about the problem. Omar was using NGINX, which I wasn’t familiar with, especially that all my work is with Microsoft products. So it was about time to understand the whole story of connection-thread approach and even driven approach in web servers’ processing method; it is the time to present what I understood in a simple and understandable way.
As I mentioned, my website had a problem in using huge amount of resources. The main reason of that high usage in connection-thread approach is that web servers like IIS and Apache are opening a process or a process for each connection, and as the number of processes increases, the server will not be able to handle the connections anymore.
In the GIF below, we can see the simplest layout of Apache processing approach: simply a request arrives; a connection opens for the request; a process is created for the request; it gets processed; and finally the response is sent back.
Everything seems fine but creating new connection for each request became costly, so a new change was introduced in Apache.
Ready connections were added, ready connection still available and waiting for a coming request. Processes were also ready for each ready connection; a (prefork) was used instead of (fork).
However, that wasn’t enough for the rapid changes happening in the web environment. Load time started to be longer after the changes in internet speed and graphics changes in the web. More load time was needed for the heavier requests. The problem became worse when the number of these concurrent heavy processes was increasing.
The next change was to use threads instead of processes, as threads are lighter than processes. The other change was to manage the connections by an event multiprocessor thread that manages idle keep-alive connections once the request has completed.
But why the problem of concurrency that I have faced is still happening for other Apache and IIS users?
In NGINX, nothing of that is happening.Instead of creating processes or threads, a connection makes an event occur; events create objects (file descriptors) which are solved by only one worker. Every request creates a state machine (set of instructions) which represents required steps to complete the request. The worker is as if s/he is one of those super chess players who play tens or hundreds of chess matches at the same time; s/he would have one move with each of the opponents and then s/he would continue the loop.
State machine is one of the simplest forms of PCs; it can be seen in ATMs, toys or the simple gates in the metro stations. When you perform an action you will change the state of the machine until it reaches the end. An analogy could be when you put a coin in the metro station gate, the gate opens, and as you pass its state will return back to the initial state (locked). Applying these simple concepts in large complex processors is very effective.
References:
- https://www.nginx.com/blog/inside-nginx-how-we-designed-for-performance-scale/
- https://www.nginx.com/resources/library/infographic-inside-nginx/
- http://www.aosabook.org/en/nginx.html
- https://www.nginx.com/blog/nginx-vs-apache-our-view/