A web server is computer software and underlying hardware that accepts requests via HTTP (the network protocol created to distribute web content) or its secure variant HTTPS. A user agent, commonly a web browser or web crawler, initiates communication by making a request for a web page or other resource using HTTP, and the server responds with the content of that resource or an error message. A web server can also accept and store resources sent from the user agent if configured to do so.[1][2] The hardware used to run a web server can vary according to the volume of requests that it needs to handle. At the low end of the range are embedded systems, such as a router that runs a small web server as its configuration interface. A high-traffic Internet website might handle requests with hundreds of servers that run on racks of high-speed computers. A resource sent from a web server can be a preexisting file (static content) available to the web server, or it can be generated at the time of the request (dynamic content) by another program that communicates with the server software. The former usually can be served faster and can be more easily cached for repeated requests, while the latter supports a broader range of applications. Technologies such as REST and SOAP, which use HTTP as a basis for general computer-to-computer communication, as well as support for WebDAV extensions, have extended the application of web servers well beyond their original purpose of serving human-readable pages. First web proposal (1989) evaluated as "vague but exciting..." The world's first web server, a NeXT Computer workstation with Ethernet, 1990. The case label reads: "This machine is a server. DO NOT POWER IT DOWN!!" This is a very brief history of web server programs, so some information necessarily overlaps with the histories of the web browsers, the World Wide Web and the Internet; therefore, for the sake of the clearness and understandability, some key historical information below reported may be similar to that found also in one or more of the above-mentioned history articles. Initial WWW project (1989-1991)In March 1989, Sir Tim Berners-Lee proposed a new project to his employer CERN, with the goal of easing the exchange of information between scientists by using a hypertext system. The proposal, titled "HyperText and CERN", asked for comments and it was read by several people. In October 1990 the proposal was reformulated and enriched (having as co-author Robert Cailliau), and finally it was approved.[3] [4] [5] Between late 1990 and early 1991 the project resulted in Berners-Lee and his developers writing and testing several software libraries along with three programs, which initially ran on NeXTSTEP OS installed on NeXT workstations: [6][7][5]
Those early browsers retrieved web pages from web server(s) using a new basic communication protocol that was named HTTP 0.9. In August 1991 Tim Berner-Lee announced the birth of WWW technology and encouraged scientists to adopt and develop it.[8] Soon after, those programs, along with their source code, were made available to people interested in their usage.[6] In practice CERN informally allowed other people, including developers, etc., to play with and maybe further develop what it has been made till that moment. This was the official birth of CERN httpd. Since then Berner-Lee started promoting the adoption and the usage of those programs along with their porting to other OSs.[5] Fast and wild development (1991-1995)
In December 1991 the first web server outside Europe was installed at SLAC (U.S.A.).[7] This was a very important event because it started trans-continental web communications between web browsers and web servers. In 1991-1993 CERN web server program continued to be actively developed by www group, meanwhile, thanks to the availability of its source code and the public specifications of the HTTP protocol, many other implementations of web servers started to be developed. In April 1993 CERN issued a public official statement stating that the three components of Web software (the basic line-mode client, the web server and the library of common code), along with their source code, were put in the public domain.[11] This statement freed web server developers from any possible legal issue about the development of derivative work based on that source code (a threat that in practice never existed). At the beginning of 1994, the most notable among new web servers was NCSA httpd which ran on a variety of Unix-based OSs and could serve dynamically generated content by implementing the POST HTTP method and the CGI to communicate with external programs. These capabilities, along with the multimedia features of NCSA's Mosaic browser (also able to manage HTML FORMs in order to send data to web server) highlighted the potential of web technology for publishing and distributed computing applications. In the second half of 1994, the development of NCSA httpd stalled to the point that a group of external software developers, webmasters and other professional figures interested in that server, started to write and collect patches thanks to the NCSA httpd source code being available to public domain. At the beginning of 1995 those patches were all applied to the last release of NCSA source code and, after several tests, the Apache HTTP server project was started.[12][13] At the end of 1994 a new commercial web server, named Netsite, was released with specific features. It was the first one of many other similar products that were developed first by Netscape, then also by Sun Microsystems and finally by Oracle Corporation. In mid-1995 the first version of IIS was released, for Windows NT OS, by Microsoft. This was a notable event because marked the entry, in the field of World Wide Web technologies, of a very important commercial developer and vendor that has played and still is playing a key role on both sides (client and server) of the web. In the second half of 1995 CERN and NCSA web servers started to decline (in global percentage usage) because of the wide-spread adoption of new web servers which had a much faster development cycle along with more features, more fixes applied and more performances than the previous ones. Explosive growth and competition (1996-2014)
Sun's Cobalt Qube 3 – a computer server appliance (2002, discontinued) At the end of 1996 there were already over fifty known (different) web server software programs that were available to everybody who wanted to own an Internet domain name and/or to host websites.[15] Many of them lived only shortly and were replaced by other web servers. The publication of RFCs about protocol versions HTTP/1.0 (1996) and HTTP/1.1 (1997, 1999), forced most web servers to comply (not always completely) with those standards. The use of TCP/IP persistent connections (HTTP/1.1) required web servers both to increase a lot the maximum number of concurrent connections allowed and to improve their level of scalability. Between 1996 and 1999 Netscape Enterprise Server and Microsoft's IIS emerged among the leading commercial options whereas among the freely available and open-source programs Apache HTTP Server held the lead as the preferred server (because of its reliability and its many features). In those years there was also another commercial, highly innovative and thus notable web server called Zeus (now discontinued) that was known as one of the fastest and most scalable web servers available on market, at least till first decade of 2000s, despite its low percentage of usage. Apache resulted in the most used web server from mid-1996 to the end of 2015 when, after a few years of decline, it was surpassed initially by IIS and then by Nginx. Afterwards IIS dropped to much lower percentages of usage than Apache (see also market share). Since 2005-2006 Apache started to improve its speed and its scalability level by introducing new performance features (e.g. event MPM and new content cache).[16][17] As those new performance improvements initially were marked as experimental, they were not enabled by its users for a long time and so Apache suffered even more the competition of commercial servers and, above all, of other open-source servers which meanwhile had already achieved far superior performances (mostly when serving static content) since the beginning of their development and at the time of the Apache decline were able to offer also a long enough list of well tested advanced features. In fact, a few years after 2000 started, not only other commercial and highly competitive web servers, e.g. LiteSpeed, but also many other open-source programs, often of excellent quality and very high performances, among which should be noted Hiawatha, Cherokee HTTP server, Lighttpd, Nginx and other derived/related products also available with commercial support, emerged. Around 2007-2008 most popular web browsers increased their previous default limit of 2 persistent connections per host-domain (a limit recommended by RFC-2616) [18] to 4, 6 or 8 persistent connections per host-domain, in order to speed up the retrieval of heavy web pages with lots of images, and to mitigate the problem of the shortage of persistent connections dedicated to dynamic objects used for bi-directional notifications of events in web pages.[19] Within a year, these changes, on average, nearly tripled the maximum number of persistent connections that web servers had to manage. This trend (of increasing the number of persistent connections) definitely gave a strong impetus to the adoption of reverse proxies in front of slower web servers and it gave also one more chance to the emerging new web servers that could show all their speed and their capability to handle very high numbers of concurrent connections without requiring too many hardware resources (expensive computers with lots of CPUs, RAM and fast disks).[20] New challenges (2015 and later years)In 2015, RFCs published new protocol version [HTTP/2], and as the implementation of new specifications was not trivial at all, a dilemma arose among developers of less popular web servers (e.g. with a percentage of usage lower than 1% .. 2%), about adding or not adding support for that new protocol version.[21][22] In fact supporting HTTP/2 often required radical changes to their internal implementation due to many factors (practically always required encrypted connections, capability to distinguish between HTTP/1.x and HTTP/2 connections on the same TCP port, binary representation of HTTP messages, message priority, compression of HTTP headers, use of streams also known as TCP/IP sub-connections and related flow-control, etc.) and so a few developers of those web servers opted for not supporting new HTTP/2 version (at least in the near future) also because of these main reasons:[21][22]
Instead, developers of most popular web servers, rushed to offer the availability of new protocol, not only because they had the work force and the time to do so, but also because usually their previous implementation of SPDY protocol could be reused as a starting point and because most used web browsers implemented it very quickly for the same reason. Another reason that prompted those developers to act quickly was that webmasters felt the pressure of the ever increasing web traffic and they really wanted to install and to try - as soon as possible - something that could drastically lower the number of TCP/IP connections and speedup accesses to hosted websites.[23] In 2020–2021 the HTTP/2 dynamics about its implementation (by top web servers and popular web browsers) were partly replicated after the publication of advanced drafts of future RFC about HTTP/3 protocol. PC clients connected to a web server via Internet The following technical overview should be considered only as an attempt to give a few very limited examples about some features that may be implemented in a web server and some of the tasks that it may perform in order to have a sufficiently wide scenario about the topic. A web server program plays the role of a server in a client–server model by implementing one or more versions of HTTP protocol, often including the HTTPS secure variant and other features and extensions that are considered useful for its planned usage. The complexity and the efficiency of a web server program may vary a lot depending on (e.g.):[1]
Common featuresAlthough web server programs differ in how they are implemented, most of them offer the following common features. These are basic features that most web servers usually have.
A few other more advanced and popular features (only a very short selection) are the following ones.
Common tasksA web server program, when it is running, usually performs several general tasks, (e.g.):[1]
Read request messageWeb server programs are able:[24][25][26]
Once an HTTP request message has been decoded and verified, its values can be used to determine whether that request can be satisfied or not. This requires many other steps, including security checks. URL normalizationWeb server programs usually perform some type of URL normalization (URL found in most HTTP request messages) in order:
The term URL normalization refers to the process of modifying and standardizing a URL in a consistent manner. There are several types of normalization that may be performed, including the conversion of the scheme and host to lowercase. Among the most important normalizations are the removal of "." and ".." path segments and adding trailing slashes to a non-empty path component. URL mapping"URL mapping is the process by which a URL is analyzed to figure out what resource it is referring to, so that that resource can be returned to the requesting client. This process is performed with every request that is made to a web server, with some of the requests being served with a file, such as an HTML document, or a gif image, others with the results of running a CGI program, and others by some other process, such as a built-in module handler, a PHP document, or a Java servlet."[27] In practice, web server programs that implement advanced features, beyond the simple static content serving (e.g. URL rewrite engine, dynamic content serving), usually have to figure out how that URL has to be handled, e.g.:
One or more configuration files of web server may specify the mapping of parts of URL path (e.g. initial parts of file path, filename extension and other path components) to a specific URL handler (file, directory, external program or internal module).[28] When a web server implements one or more of the above-mentioned advanced features then the path part of a valid URL may not always match an existing file system path under website directory tree (a file or a directory in file system) because it can refer to a virtual name of an internal or external module processor for dynamic requests. URL path translation to file systemWeb server programs are able to translate an URL path (all or part of it), that refers to a physical file system path, to an absolute path under the target website's root directory.[28] Website's root directory may be specified by a configuration file or by some internal rule of the web server by using the name of the website which is the host part of the URL found in HTTP client request.[28] Path translation to file system is done for the following types of web resources:
The web server appends the path found in requested URL (HTTP request message) and appends it to the path of the (Host) website root directory. On an Apache server, this is commonly /home/www/website (on Unix machines, usually it is: /var/www/website). See the following examples of how it may result. URL path translation for a static file request Example of a static request of an existing file specified by the following URL: http://www.example.com/path/file.htmlThe client's user agent connects to www.example.com and then sends the following HTTP/1.1 request: GET /path/file.html HTTP/1.1 Host: www.example.com Connection: keep-aliveThe result is the local file system resource: /home/www/www.example.com/path/file.htmlThe web server then reads the file, if it exists, and sends a response to the client's web browser. The response will describe the content of the file and contain the file itself or an error message will return saying that the file does not exist or its access is forbidden. URL path translation for a directory request (without a static index file) Example of an implicit dynamic request of an existing directory specified by the following URL: http://www.example.com/directory1/directory2/The client's user agent connects to www.example.com and then sends the following HTTP/1.1 request: GET /directory1/directory2 HTTP/1.1 Host: www.example.com Connection: keep-aliveThe result is the local directory path: /home/www/www.example.com/directory1/directory2/The web server then verifies the existence of the directory and if it exists and it can be accessed then tries to find out an index file (which in this case does not exist) and so it passes the request to an internal module or a program dedicated to directory listings and finally reads data output and sends a response to the client's web browser. The response will describe the content of the directory (list of contained subdirectories and files) or an error message will return saying that the directory does not exist or its access is forbidden. URL path translation for a dynamic program request For a dynamic request the URL path specified by the client should refer to an existing external program (usually an executable file with a CGI) used by the web server to generate dynamic content.[29] Example of a dynamic request using a program file to generate output: http://www.example.com/cgi-bin/forum.php?action=view&orderby=thread&date=2021-10-15The client's user agent connects to www.example.com and then sends the following HTTP/1.1 request: GET /cgi-bin/forum.php?action=view&ordeby=thread&date=2021-10-15 HTTP/1.1 Host: www.example.com Connection: keep-aliveThe result is the local file path of the program (in this example, a PHP program): /home/www/www.example.com/cgi-bin/forum.phpThe web server executes that program, passing in the path-info and the query string action=view&orderby=thread&date=2021-10-15 so that the program has the info it needs to run. (In this case, it will return an HTML document containing a view of forum entries ordered by thread from October 15th, 2021). In addition to this, the web server reads data sent from the external program and resends that data to the client that made the request. Manage request messageOnce a request has been read, interpreted, and verified, it has to be managed depending on its method, its URL, and its parameters, which may include values of HTTP headers. In practice, the web server has to handle the request by using one of these response paths:[28]
Serve static contentPC clients communicating via network with a web server serving static content only. If a web server program is capable of serving static content and it has been configured to do so, then it is able to send file content whenever a request message has a valid URL path matching (after URL mapping, URL translation and URL redirection) that of an existing file under the root directory of a website and file has attributes which match those required by internal rules of web server program.[28] That kind of content is called static because usually it is not changed by the web server when it is sent to clients and because it remains the same until it is modified (file modification) by some program. NOTE: when serving static content only, a web server program usually does not change file contents of served websites (as they are only read and never written) and so it suffices to support only these HTTP methods:
Response of static file content can be sped up by a file cache. Directory index filesIf a web server program receives a client request message with an URL whose path matches one of an existing directory and that directory is accessible and serving directory index file(s) is enabled then a web server program may try to serve the first of known (or configured) static index file names (a regular file) found in that directory; if no index file is found or other conditions are not met then an error message is returned. Most used names for static index files are: index.html, index.htm and Default.htm. Regular filesIf a web server program receives a client request message with an URL whose path matches the file name of an existing file and that file is accessible by web server program and its attributes match internal rules of web server program, then web server program can send that file to client. Usually, for security reasons, most web server programs are pre-configured to serve only regular files or to avoid to use special file types like device files, along with symbolic links or hard links to them. The aim is to avoid undesirable side effects when serving static web resources.[30] Serve dynamic contentPC clients communicating via network with a web server serving static and dynamic content. If a web server program is capable of serving dynamic content and it has been configured to do so, then it is able to communicate with the proper internal module or external program (associated with the requested URL path) in order to pass to it parameters of client request; after that, web server program reads from it its data response (that it has generated, often on the fly) and then it resends it to the client program who made the request.[citation needed] NOTE: when serving static and dynamic content, a web server program usually has to support also the following HTTP method in order to be able to safely receive data from client(s) and so to be able to host also websites with interactive form(s) that may send large data sets (e.g. lots of data entry or file uploads) to web server / external programs / modules:
In order to be able to communicate with its internal modules and/or external programs, a web server program must have implemented one or more of the many available gateway interface(s) (see also Web Server Gateway Interfaces used for dynamic content). The three standard and historical gateway interfaces are the following ones. CGI An external CGI program is run by web server program for each dynamic request, then web server program reads from it the generated data response and then resends it to client.SCGI An external SCGI program (it usually is a process) is started once by web server program or by some other program / process and then it waits for network connections; every time there is a new request for it, web server program makes a new network connection to it in order to send request parameters and to read its data response, then network connection is closed.FastCGI An external FastCGI program (it usually is a process) is started once by web server program or by some other program / process and then it waits for a network connection which is established permanently by web server; through that connection are sent the request parameters and read data responses.Directory listingsDirectory listing dynamically generated by a web server. A web server program may be capable to manage the dynamic generation (on the fly) of a directory index list of files and sub-directories.[31] If a web server program is configured to do so and a requested URL path matches an existing directory and its access is allowed and no static index file is found under that directory then a web page (usually in HTML format), containing the list of files and/or subdirectories of above mentioned directory, is dynamically generated (on the fly). If it cannot be generated an error is returned. Some web server programs allow the customization of directory listings by allowing the usage of a web page template (an HTML document containing placeholders, e.g. $(FILE_NAME), $(FILE_SIZE), etc., that are replaced with the field values of each file entry found in directory by web server), e.g. index.tpl or the usage of HTML and embedded source code that is interpreted and executed on the fly, e.g. index.asp, and / or by supporting the usage of dynamic index programs such as CGIs, SCGIs, FGCIs, e.g. index.cgi, index.php, index.fcgi. Usage of dynamically generated directory listings is usually avoided or limited to a few selected directories of a website because that generation takes much more OS resources than sending a static index page. The main usage of directory listings is to allow the download of files (usually when their names, sizes, modification date-times or file attributes may change randomly / frequently) as they are, without requiring to provide further information to requesting user.[32] Program or module processingAn external program or an internal module (processing unit) can execute some sort of application function that may be used to get data from or to store data to one or more data repositories, e.g.:[citation needed]
A processing unit can return any kind of web content, also by using data retrieved from a data repository, e.g.:[citation needed]
In practice whenever there is content that may vary, depending on one or more parameters contained in client request or in configuration settings, then, usually, it is generated dynamically. Send response messageWeb server programs are able to send response messages as replies to client request messages.[24] An error response message may be sent because a request message could not be successfully read or decoded or analyzed or executed.[25] NOTE: the following sections are reported only as examples to help to understand what a web server, more or less, does; these sections are by any means neither exhaustive nor complete. Error messageA web server program may reply to a client request message with many kinds of error messages, anyway these errors are divided mainly in two categories:
When an error response / message is received by a client browser, then if it is related to the main user request (e.g. an URL of a web resource such as a web page) then usually that error message is shown in some browser window / message. URL authorizationA web server program may be able to verify whether the requested URL path:[35]
If the authorization / access rights feature has been implemented and enabled and access to web resource is not granted, then, depending on the required access rights, a web server program:
URL redirectionA web server program may have the capability of doing URL redirections to new URLs (new locations) which consists in replying to a client request message with a response message containing a new URL suited to access a valid or an existing web resource (client should redo the request with the new URL).[36] URL redirection of location is used:[36]
Example 1: a URL path points to a directory name but it does not have a final slash '/' so web server sends a redirect to client in order to instruct it to redo the request with the fixed path name.[31] From: Example 2: a whole set of documents has been moved inside website in order to reorganize their file system paths. From: Example 3: a whole set of documents has been moved to a new website and now it is mandatory to use secure HTTPS connections to access them. From: Above examples are only a few of the possible kind of redirections. Successful messageA web server program is able to reply to a valid client request message with a successful message, optionally containing requested web resource data.[37] If web resource data is sent back to client, then it can be static content or dynamic content depending on how it has been retrieved (from a file or from the output of some program / module). Content cacheIn order to speed up web server responses by lowering average HTTP response times and hardware resources used, many popular web servers implement one or more content caches, each one specialized in a content category.[38][39] Content is usually cached by its origin, e.g.:
File cacheHistorically, static contents found in files which had to be accessed frequently, randomly and quickly, have been stored mostly on electro-mechanical disks since mid-late 1960s / 1970s; regrettably reads from and writes to those kind of devices have always been considered very slow operations when compared to RAM speed and so, since early OSs, first disk caches and then also OS file cache sub-systems were developed to speed up I/O operations of frequently accessed data / files. Even with the aid of an OS file cache, the relative / occasional slowness of I/O operations involving directories and files stored on disks became soon a bottleneck in the increase of performances expected from top level web servers, specially since mid-late 1990s, when web Internet traffic started to grow exponentially along with the constant increase of speed of Internet / network lines. The problem about how to further efficiently speed-up the serving of static files, thus increasing the maximum number of requests/responses per second (RPS), started to be studied / researched since mid 1990s, with the aim to propose useful cache models that could be implemented in web server programs.[40][41] In practice, nowadays, many popular / high performance web server programs include their own userland file cache, tailored for a web server usage and using their specific implementation and parameters.[42][43][44] The wide spread adoption of RAID and/or fast solid-state drives (storage hardware with very high I/O speed) has slightly reduced but of course not eliminated the advantage of having a file cache incorporated in a web server. Dynamic cacheDynamic content, output by an internal module or an external program, may not always change very frequently (given a unique URL with keys / parameters) and so, maybe for a while (e.g. from 1 second to several hours or more), the resulting output can be cached in RAM or even on a fast disk.[45] The typical usage of a dynamic cache is when a website has dynamic web pages about news, weather, images, maps, etc. that do not change frequently (e.g. every n minutes) and that are accessed by a huge number of clients per minute / hour; in those cases it is useful to return cached content too (without calling the internal module or the external program) because clients often do not have an updated copy of the requested content in their browser caches.[46] Anyway, in most cases those kind of caches are implemented by external servers (e.g. reverse proxy) or by storing dynamic data output in separate computers, managed by specific applications (e.g. memcached), in order to not compete for hardware resources (CPU, RAM, disks) with web server(s).[47][48] Kernel-mode and user-mode web serversA web server software can be either incorporated into the OS and executed in kernel space, or it can be executed in user space (like other regular applications). Web servers that run in kernel mode (usually called kernel space web servers) can have direct access to kernel resources and so they can be, in theory, faster than those running in user mode; anyway there are disadvantages in running a web server in kernel mode, e.g.: difficulties in developing (debugging) software whereas run-time critical errors may lead to serious problems in OS kernel. Web servers that run in user-mode have to ask the system for permission to use more memory or more CPU resources. Not only do these requests to the kernel take time, but they might not always be satisfied because the system reserves resources for its own usage and has the responsibility to share hardware resources with all the other running applications. Executing in user mode can also mean using more buffer/data copies (between user-space and kernel-space) which can lead to a decrease in the performance of a user-mode web server. Nowadays almost all web server software is executed in user mode (because many of the aforementioned small disadvantages have been overcome by faster hardware, new OS versions, much faster OS system calls and new optimized web server software). See also comparison of web server software to discover which of them run in kernel mode or in user mode (also referred as kernel space or user space). To improve the user experience (on client / browser side), a web server should reply quickly (as soon as possible) to client requests; unless content response is throttled (by configuration) for some type of files (e.g. big or huge files), also returned data content should be sent as fast as possible (high transfer speed). In other words, a web server should always be very responsive, even under high load of web traffic, in order to keep total user's wait (sum of browser time + network time + web server response time) for a response as low as possible. Performance metricsFor web server software, main key performance metrics (measured under vary operating conditions) usually are at least the following ones (i.e.):[49][50]
Among the operating conditions, the number (1 .. n) of concurrent client connections used during a test is an important parameter because it allows to correlate the concurrency level supported by web server with results of the tested performance metrics. Software efficiencyThe specific web server software design and model adopted (e.g.):
... and other programming techniques, such as (e.g.):
... used to implement a web server program, can bias a lot the performances and in particular the scalability level that can be achieved under heavy load or when using high end hardware (many CPUs, disks and lots of RAM). In practice some web server software models may require more OS resources (specially more CPUs and more RAM) than others to be able to work well and so to achieve target performances. Operating conditionsThere are many operating conditions that can affect the performances of a web server; performance values may vary depending on (i.e.):
BenchmarkingPerformances of a web server are typically benchmarked by using one or more of the available automated load testing tools. A web server (program installation) usually has pre-defined load limits for each combination of operating conditions, also because it is limited by OS resources and because it can handle only a limited number of concurrent client connections (usually between 2 and several tens of thousands for each active web server process, see also the C10k problem and the C10M problem). When a web server is near to or over its load limits, it gets overloaded and so it may become unresponsive. Causes of overloadAt any time web servers can be overloaded due to one or more of the following causes (e.g.).
Symptoms of overloadThe symptoms of an overloaded web server are usually the following ones (e.g.).
Anti-overload techniquesTo partially overcome above average load limits and to prevent overload, most popular websites use common techniques like the following ones (e.g.).
Caveats about using HTTP/2 and HTTP/3 protocols Even if newer HTTP (2 and 3) protocols usually generate less network traffic for each request / response data, they may require more OS resources (i.e. RAM and CPU) used by web server software (because of encrypted data, lots of stream buffers and other implementation details); besides this, HTTP/2 and maybe HTTP/3 too, depending also on settings of web server and client program, may not be the best options for data upload of big or huge files at very high speed because their data streams are optimized for concurrency of requests and so, in many cases, using HTTP/1.1 TCP/IP connections may lead to better results / higher upload speeds (your mileage may vary).[55][56] Chart: Market share of all sites for most popular web servers 2005–2021 Chart: Market share of all sites for most popular web servers 1995–2005 Below are the latest statistics of the market share of all sites of the top web servers on the Internet by Netcraft.
NOTE: (*) percentage rounded to integer number, because its decimal values are not publicly reported by source page (only its rounded value is reported in graph).
Standard Web Server Gateway Interfaces used for dynamic contents:
A few other Web Server Interfaces (server or programming language specific) used for dynamic contents:
Page 2HTTP/2 (originally named HTTP/2.0) is a major revision of the HTTP network protocol used by the World Wide Web. It was derived from the earlier experimental SPDY protocol, originally developed by Google.[1][2] HTTP/2 was developed by the HTTP Working Group (also called httpbis, where "bis" means "twice") of the Internet Engineering Task Force (IETF).[3][4][5] HTTP/2 is the first new version of HTTP since HTTP/1.1, which was standardized in RFC 2068 in 1997. The Working Group presented HTTP/2 to the Internet Engineering Steering Group (IESG) for consideration as a Proposed Standard in December 2014,[6][7] and IESG approved it to publish as Proposed Standard on February 17, 2015 (and was updated in February 2020 in regard to TLS 1.3).[8][9] The HTTP/2 specification was published as RFC 7540 on May 14, 2015.[10] The standardization effort was supported by Chrome, Opera, Firefox,[11] Internet Explorer 11, Safari, Amazon Silk, and Edge browsers.[12] Most major browsers had added HTTP/2 support by the end of 2015.[13] About 97% of web browsers used have the capability.[14] As of October 2021[update], 47% (after topping out at just over 50%) of the top 10 million websites supported HTTP/2.[15] Its successor is HTTP/3, a major revision that builds on the concepts established by HTTP/2.[2][16][17][18] The working group charter mentions several goals and issues of concern:[4] The proposed changes do not require any changes to how existing web applications work, but new applications can take advantage of new features for increased speed.[19] HTTP/2 leaves all of HTTP/1.1's high-level semantics, such as methods, status codes, header fields, and URIs, the same. What is new is how the data is framed and transported between the client and the server.[19] Websites that are efficient minimize the number of requests required to render an entire page by minifying (reducing the amount of code and packing smaller pieces of code into bundles, without reducing its ability to function) resources such as images and scripts. However, minification is not necessarily convenient nor efficient and may still require separate HTTP connections to get the page and the minified resources. HTTP/2 allows the server to "push" content, that is, to respond with data for more queries than the client requested. This allows the server to supply data it knows a web browser will need to render a web page, without waiting for the browser to examine the first response, and without the overhead of an additional request cycle.[20] Additional performance improvements in the first draft of HTTP/2 (which was a copy of SPDY) come from multiplexing of requests and responses to avoid some of the head-of-line blocking problem in HTTP 1 (even when HTTP pipelining is used), header compression, and prioritization of requests.[21] However, as HTTP/2 runs on top of a single TCP connection there is still potential for head-of-line blocking to occur if TCP packets are lost or delayed in transmission.[22]
HTTP/2 no longer supports HTTP/1.1's chunked transfer encoding mechanism, as it provides its own, more efficient, mechanisms for data streaming.[23] SPDY (pronounced like "speedy") was a previous HTTP-replacement protocol developed by a research project spearheaded by Google.[24] Primarily focused on reducing latency, SPDY uses the same TCP pipe but different protocols to accomplish this reduction. The basic changes made to HTTP/1.1 to create SPDY included: "true request pipelining without FIFO restrictions, message framing mechanism to simplify client and server development, mandatory compression (including headers), priority scheduling, and even bi-directional communication".[25] The HTTP Working Group considered Google's SPDY protocol, Microsoft's HTTP Speed+Mobility proposal (SPDY based),[24] and Network-Friendly HTTP Upgrade.[26] In July 2012, Facebook provided feedback on each of the proposals and recommended HTTP/2 be based on SPDY.[27] The initial draft of HTTP/2 was published in November 2012 and was based on a straight copy of SPDY.[28] The biggest difference between HTTP/1.1 and SPDY was that each user action in SPDY is given a "stream ID", meaning there is a single TCP channel connecting the user to the server. SPDY split requests into either control or data, using a "simple to parse binary protocol with two types of frames".[25][29] SPDY showed evident improvement over HTTP, with a new page load speedup ranging from 11% to 47%.[30] The development of HTTP/2 used SPDY as a jumping-off point. Among the many detailed differences between the protocols, the most notable is that HTTP/2 uses a fixed Huffman code-based header compression algorithm, instead of SPDY's dynamic stream-based compression. This helps to reduce the potential for compression oracle attacks on the protocol, such as the CRIME attack.[29] On February 9, 2015, Google announced plans to remove support for SPDY in Chrome in favor of support for HTTP/2.[31] That took effect, starting with Chrome 51.[32][33] HTTP/2 is defined both for HTTP URIs (i.e. without TLS encryption, a configuration which is abbreviated in h2c) and for HTTPS URIs (over TLS using ALPN extension[50] where TLS 1.2 or newer is required, a configuration which is abbreviated in h2).
Although the standard itself does not require usage of encryption,[51] all major client implementations (Firefox,[52] Chrome, Safari, Opera, IE, Edge) have stated that they will only support HTTP/2 over TLS, which makes encryption de facto mandatory.[53] The FreeBSD and Varnish developer Poul-Henning Kamp asserts that the standard was prepared on an unrealistically short schedule, ruling out any basis for the new HTTP/2 other than the SPDY protocol and resulting in other missed opportunities for improvement. Kamp criticizes the protocol itself for being inconsistent and having needless, overwhelming complexity. He also states that the protocol violates the protocol layering principle, for example by duplicating flow control that belongs in the transport layer (TCP). He also suggested that the new protocol should have removed HTTP Cookies, introducing a breaking change.[54] Initially, some members[who?] of the Working Group tried to introduce an encryption requirement in the protocol. This faced criticism.
Critics stated that encryption has non-negligible computing costs and that many HTTP applications actually have no need for encryption and their providers have no desire to spend additional resources on it. Encryption proponents have stated that this encryption overhead is negligible in practice.[55] Poul-Henning Kamp has criticized the IETF for hastily standardizing Google's SPDY prototype as HTTP/2 due to political considerations.[54][56][57] The criticism of the agenda of mandatory encryption within the existing certificate framework is not new, nor is it unique to members of the open-source community – a Cisco employee stated in 2013 that the present certificate model is not compatible with small devices like routers, because the present model requires not only annual enrollment and remission of non-trivial fees for each certificate, but must be continually repeated on an annual basis.[58] In the end the Working Group did not reach consensus over the mandatory encryption,[51] although most client implementations require it, which makes encryption a de facto requirement.
The HTTP/2 protocol also faced criticism for not supporting opportunistic encryption, a measure against passive monitoring similar to the STARTTLS mechanism that has long been available in other Internet protocols like SMTP. Critics have stated that the HTTP/2 proposal goes in violation of IETF's own RFC 7258 "Pervasive Monitoring Is an Attack", which also has a status of Best Current Practice 188.[59] RFC7258/BCP188 mandates that passive monitoring be considered as an attack, and protocols designed by IETF should take steps to protect against passive monitoring (for example, through the use of opportunistic encryption). A number of specifications for opportunistic encryption of HTTP/2 have been provided,[60][61][62] of which draft-nottingham-http2-encryption was adopted as an official work item of the working group, leading to the publication of RFC 8164 in May 2017.
Although the design of HTTP/2 effectively addresses the HTTP-transaction-level head-of-line blocking problem by allowing multiple concurrent HTTP transactions, all those transactions are multiplexed over a single TCP connection, meaning that any packet-level head-of-line blocking of the TCP stream simultaneously blocks all transactions being accessed via that connection. This head-of-line blocking in HTTP/2 is now widely regarded as a design flaw, and much of the effort behind QUIC and HTTP/3 has been devoted to reduce head-of-line blocking issues.[63][64]
|