What is an in­ter­na­tion­al­ised domain name (IDN)?

According to the in­ter­na­tion­al tele­com­mu­nic­a­tions union (ITU), more than three billion people use the World Wide Web and in­creas­ingly so in their mother tongues. This change was in part brought on by the in­tro­duc­tion of in­ter­na­tion­al domain names in 2003. We’ll explain how IDN domains work.

What is an in­ter­na­tion­al­ised domain name (IDN)?

The IETF (Internet En­gin­eer­ing Task Force) refers to IDNs as domain names that contain special char­ac­ters that are not part of the Latin alphabet, such as umlauts or char­ac­ters from other alphabets. However, the Domain Name System (DNS), which is re­spons­ible for trans­lat­ing URLs into IP addresses, cannot un­der­stand these domain names. The DNS is based on the limited standard character set ASCII.

In order to make IDNs un­der­stand­able for the DNS as well as other internet protocols, the internet standard In­ter­na­tion­al­ising Domain Names in Ap­plic­a­tions (IDNA) was created in 2003. This defines a stand­ard­ised trans­la­tion from Unicode to ASCII, therefore enabling the use of non-ASCII char­ac­ters in domain names.

How does IDNA work?

Much of the internet’s in­fra­struc­ture is only supported by the ASCII character set. In order to make sure that in­ter­na­tion­al domain names can be processed, each IDN that’s available in Unicode is trans­lated into an ACE string, which is based on ASCII. Following this, URLs featuring char­ac­ters with accents or umlauts are displayed. The server, on the other hand, continues to process the addresses as ASCII com­pat­ible. This procedure is specified in the IDNA2003 internet standard and in the IDNA2008 revision, which was approved in 2010. Trans­lat­ing from Unicode to ASCII occurs client-side (in the browser, email program, etc.) and is based on a stand­ard­ised coding process called Punycode.

Punycode

The RFC 3492-stand­ard­ised Punycode was developed for clearly dis­play­ing Unicode character strings as ASCII symbols without loss of quality. All non-ASCII char­ac­ters are removed from the domain name, encoded and separated with a hyphen. This code sequence contains in­form­a­tion about the Unicode symbol in question as well as its position in the domain name. Ad­di­tion­ally, each ACE string created in this way is labelled with the prefix xn–. This clarifies to the reader that the character sequence is an IDN that has been encoded according to IDNA and Punycode standards. See our article on Punycode for a detailed ex­plan­a­tion of the encoding process as well as some examples.

Dif­fer­ences between IDNA2003 and IDNA2008

For the original 2003 procedure, in­ter­na­tion­al­ised URLs were nor­m­al­ised prior to Punycode encoding using the nameprep method. This method changed capital letters into lowercase letters, removed control char­ac­ters and trans­ferred equi­val­ent char­ac­ters into a unified form. Nameprep was removed from this process when IDNA2008 was in­tro­duced. Now, IDNA does not specify any nor­m­al­isa­tion. Instead, it re­com­mends an algorithm that converts capital letters into lowercase ones.

This adaption also ac­com­mod­ates users in the German-speaking world, since the Unicode character ‘ß’, which is common in Germany, was ori­gin­ally defined as the equi­val­ent of ‘ss’ according to IDNA2003. Domains such as www.fußball-ergebnisse.de were thus auto­mat­ic­ally nor­m­al­ised to www.fussball-ergebnisse.de in the nameprep process. This is no longer the case since IDNA2008 came into the picture. Since 2010, the ‘ß’ is correctly in­ter­preted as ‘Latin small letter sharp s’ and can be re­gistered as part of an IDN domain.

In addition, around 8,000 char­ac­ters that were possible in domain names under IDNA2003 are no longer supported under IDNA2008. Four char­ac­ters including ‘ß’ are in­ter­preted dif­fer­ently since the standard was revised. For a detailed dis­cus­sion of the dif­fer­ences between IDNA2003 and IDNA2008, see Unicode Technical Standard #46. The following table provides a summary of the main dif­fer­ences:

What problems are there with IDNs?

By now, all common internet programs should be able to un­der­stand IDN. However, problems with in­ter­na­tion­al­ised domain names sometimes occur because the switch from IDNA2003 to IDNA2008 has not yet been con­sist­ently im­ple­men­ted. One example that’s prob­lem­at­ic for German is the different in­ter­pret­a­tion of ‘ß’. Since IDNA2003 com­pulsor­ily converts ‘ß’ to ‘ss’, special ß domains that can be re­gistered according to IDNA2008 are often not dis­cov­er­able for systems that convert according to the outdated standard. Instead, users are directed to the cor­res­pond­ing domain con­tain­ing ‘ss’. This problem can be cir­cum­ven­ted by website operators re­gis­ter­ing both variants and re­dir­ect­ing the second domain to the pri­or­it­ised spelling using a domain redirect.