What is a rewrite engine?

A rewrite engine is a component of web server software that allows you to rewrite or redirect uniform resource locators (URLs). The most popular rewrite engine is the Apache HTTP server’s mod_rewrite. There are other web servers, such as nginx or lighttpd, that provide similar functions.

Some content management systems generate cumbersome URLs. This software component is used to convert them into user-friendly URLs. The reasons for this are obvious: technical URLs such as

"http://example.com/a/index.php?title=pagetitle"

aren’t easy to remember; a rewrite engine enables the URL to be written in a much more intuitive way:

"http://example.com/article/pagetitle"

An internet user can also use this URL to access a corresponding web page. If a request like this is received by the web server, the rewrite engine automatically converts the URL into the internal one used by the server

"http://example.com/a/index.php?title=pagetitle"

The rewrite engine creates an abstraction layer between the URLs used internally by the web project and the URLs publically displayed online. This makes it possible to provide a user-friendly, consistent address scheme, regardless of internal technical requirements.

Internally, the dynamic, parameterised address can continue to be used while users from the internet access the web project via a seemingly static address. This function has the advantage that externally presented URLs remain valid even if internal changes are made to the file hierarchy.

In addition, website operators can use the rewrite engine to implement address redirects, which can be linked to specific conditions. In effect, it is possible to set up redirects based on the user agent string or the IP address of the requesting client, in order to implement geo targeting or to have targeted, optimised websites appear on different devices. A 301 redirect is generally used, which ensures that only one version of the website is stored in the search engine’s index, despite the parallel operation of additional mobile websites or different language versions.

Website operators should make sure they stay away from cloaking practices, which is where optimised sites are specifically created and shown to search engine crawlers in order to improve rankings.

Examples of use

The rewrite engine provides various commands for manipulating URLs, which can be identified as rules in different parts of the web server software. Mod_rewrite, the rewrite engine of the Apache web server, can be used in a directory container within httpd.conf, in a virtual host section or within an .htaccess file. With nginx, URL rewriting is recorded in the configuration file /etc/nginx/nginx.conf. With lighttpd, the /etc/lighttpd.conf file is available in the vHost configuration.

In order to rewrite the dynamic URL "http://example.com/a/index.php?title=pagetitle" via the rewrite engine into the static URL "http://example.com/article/pagetitle", different commands are used with Apache, nginx, and lighttpd web servers.

Apache’s rewrite engine

To use mod_rewrite with Apache, the rewrite engine needs to be activated with the RewriteEngine ondirective. This is followed by the RewriteRule, which defines instructions for URL rewriting using regular expressions (Regex):

RewriteEngine on
RewriteRule ^/article/(.*)$ /a/index.php?title=$1

If rewriting needs to be defined by a RewriteRule, then it can be broken down into two parameters: the search pattern and the target pattern.

  • Search pattern: this pattern describes the URLs that are to be redirected. A certain condition is defined in the form of a search pattern. If this condition is fulfilled, the URL can be rewritten according to the target pattern. In the current example, the search pattern would be the following section of the RewriteRule: ^/article/(.*)$.
  • Target pattern: this parameter describes the target URL. If the redirection is configured at server level, the complete URL is replaced. At directory level in the .htaccess file or within the httpd.conf, only the path from the current directory onwards is replaced. In this example, the target pattern includes this section of the RewriteRule: /a/index.php?title=$1.

An explanation of the expressions regularly used in the examples can be seen in this table:

Regular expression Explanation
^ Beginning of string.
$ End of string.
(.*) A place holder for any string within a URL. The parentheses store the string in a variable.
$1 A variable that allows access to cached values that are stored within brackets.

The RewriteRule ^/article/(.*)$ /a/index.php?title=$1 defines the rule that all URLs beginning with the string /article/ should be converted to the dynamic URL scheme /a/index.php?title=$1, where $1 is the string that corresponds to anything that comes after /article/, captured by the place holder (.*).

If a user enters the static URL "http://example.com/article/pagetitle" into the web browser, the web server converts this internally to the dynamic URL "http://example.com/a/index.php?title=pagetitle", based on mod_rewrite. This is invisible to the user. In this case, the place holder (.*) and the variable $1 correspond to the 'pagetitle' string.

If URL rewriting is to be linked to certain options that control the behaviour of the mod_rewrite, they are listed in square brackets after the RewriteRule, and separated by commas, if there are several. In this way, external redirecting via HTTP status code can also be implemented. The following table shows a selection of options for the RewriteRule. A complete list can be found on the official website of the Apache Software Foundation.

Option Flag Function
Redirect R The [R] flag causes an HTTP redirect to be issued to the browser with the 302 status code. To use another status code, append it with an equal sign to the flag, e.g. [R=301].
Forbidden F This instructs the webserver to send the web browser a 403 (forbidden) HTTP status code.
Gone G The webserver sends the browser the HTTP status code 410 (gone) and shows that a resource that used to be available is no longer available.
Last L This flag causes mod_rewrite to stop processing the rule set.
Nocase NC When verifying whether a URL meets the rewriting conditions, upper and lower case do not matter.
Chain C The next RewriteRule is only considered if the current condition is met.

Based on such an option, an external forwarding via HTTP status code could be implemented as follows:

RewriteEngine On
RewriteRule ^oldpage.html$ /newpage.html [R=301]

In addition to RewriteRules, mod_rewrite can also be used to define RewriteConds which can be used by website operators to set additional conditions that must be met for URL rewriting.

The syntax of a RewriteCond corresponds to the following structure and is noted before the RewriteRule:

RewriteCond TESTSTRING CONDITION

The test string typically contains server variables which are defined by percent signs and curly brackets such as %{HTTP_HOST}. The following table shows a selection of server variables.

Server variable Explanation
HTTP_USER_AGENT Refers to the client used for server access. The variable is generally used to provide an optimised website to various web browsers.
HTTP_HOST Refers to the hostname. This can include values such as domain.com, subdomain.domain.com, or the IP address.
SERVER_PORT Refers to the addressed port (e.g. 80 for HTTP or 443 for HTTPS). The variable allows website operators to redirect visitors to a secure connection.
REMOTE_ADDR Refers to the IP address of the user accessing the web server. This variable is sometimes used to block spam attacks.

The following example shows a RewriteCond that binds a subsequent RewriteRule to the user’s IP address:

RewriteCond %{REMOTE_ADDR} 173.45.68.79 

URL rewriting with nginx

The webserver, nginx, also supports URL rewriting natively. This can be implemented by using regular expressions. In order to convert URLs, the rewriting command is simply inserted into the web server configuration file /etc/nginx/nginx.conf according to the nginx syntax in a { [...] } block:

location /article {
 rewrite ^/article/(.*)$ /index.php?title=$1 last;
}

The location /article part tells website operators that URL rewriting is referencing the subdirectory article. The regular expressions for rewriting correspond to those that are also used with the Apache web server and are initiated with the rewrite command. The last flag indicates that the rewriting should be done internally and therefore without being redirected. The following flags are available for temporary or permanent redirects:

Flag Explanation
last URLs are rewritten internally. There is no redirecting.
redirect The user is temporarily redirected to the new URL by a 302 redirect.
permanent The user is permanently redirected to the new URL by a 301 redirect.

If no flag is set, then nginx automatically assigns the HTTP error code 500.

Rewrite with lighttpd

With lighttpd, URL rewriting is implemented on the basis of the url.rewrite-TYPE function. The place holder TYPE stands for various rewriting configuration options:

Rewriting configuration options Explanation
url.rewrite-once One-time URL rewriting. If the search pattern is found and the corresponding URL is rewritten to the target pattern, no more rewriting needs to take place.
url.rewrite-repeat In contrast to url.rewrite-once, url.rewrite-repeat can be followed by further rewriting.

The syntax basically follows the same pattern as with Apache when rewriting with lighttpd since the same regular expressions are used:

url.rewrite-once = (
 "^/article/(.*)$" => "/index.php?title=$1"
)

If an external redirection is to take place instead of an internal one in lighttpd, the rewriting module isn’t used, but rather a redirect module where URLs pass through the rewrite, then the redirect module.

Rewrite with Microsoft IIS

The platform Microsoft Internet Information Services (IIS) doesn’t natively have a rewrite engine. However, this can be added later on to the webserver with the Modul IIS URL Rewrite 2.0. This also allows Microsoft users to make URLs available to their website visitors, without having to interfere with the internal file management. After downloading, the URL rewriting extension is integrated directly into the IIS Manager interface where RewritingRules are entered via a graphical user interface. IIS URL Rewrite 2.0 uses regular expressions to define URL search and target patterns.

URL rewriting and search engine optimisation

Through rewriting, parameterised URLs can be converted into meaningful URLs, which makes them more user-friendly. Because of this, the functions of mod_rewrite and corresponding implementations in other web server systems are often discussed when talking about search engine optimisation. It’s been a topic of discussion for a while as to whether user-friendly URLs count as ranking factors since there is no clear evidence for a direct relation. However, website operators are likely to benefit from indirect efforts. In contract to cryptic parameters, rewritten URLs allow internet users to understand where a link leads. Rewriting can therefore be used to build trust and possibly increase click-through numbers. In addition, search terms in URLs are displayed in bold on the search results page, which can provide an additional incentive for clicks.


Wait! We’ve got something for you!
Get your .co.uk domain now, the first year is free.

Enter the web address of your choice in the search bar to check its availability.
12 month for £0/year
then £10/year