IDN, Punycode (xn--) and Homograph Attacks

The problem DNS had to solve

The DNS protocol carries labels as bytes, and long-standing host-name rules limit labels to letters, digits and hyphens. Rather than change every resolver, mail server and registry, the Internet standards body defined a way to represent any Unicode label using only those safe characters. Software that understands the scheme shows the Unicode form to people and uses the ASCII form on the wire.

Two forms of the same name exist:

• The U-label is what a person sees: bücher.example.
• The A-label is what DNS carries: xn--bcher-kva.example.

The prefix xn-- marks a label that is punycode-encoded. The mapping is reversible, so every A-label corresponds to exactly one U-label.

Worked examples (computed)

These conversions were produced with Python's built-in idna codec. Try them yourself with "bücher".encode("idna").

Unicode label	Script	A-label
bücher	Latin with diacritic	xn--bcher-kva
münchen	Latin with diacritic	xn--mnchen-3ya
café	Latin with diacritic	xn--caf-dma
mañana	Latin with diacritic	xn--maana-pta
пример	Cyrillic	xn--e1afmkfd
例え	Han and Hiragana	xn--r8jz45g
ひらがな	Hiragana	xn--v8j0cwa6g
pаypal (Cyrillic second letter)	Mixed Latin and Cyrillic	xn--pypal-4ve

Reading a punycode label

Look at xn--bcher-kva. After the prefix, the label contains the ASCII letters from the original (bcher), a hyphen as separator, and a suffix (kva) that encodes where the non-ASCII character goes and which character it is. For a label with no ASCII characters at all, such as xn--e1afmkfd, there is no separator and the whole payload is the encoded part. The encoding of the suffix uses a variable-length base-36 representation of position and code-point differences, which is why even short non-Latin labels turn into a handful of opaque letters.

The last row above is the one that matters for security. The visible text pаypal contains U+0430, CYRILLIC SMALL LETTER A, instead of Latin "a". In most fonts it is indistinguishable, yet the A-label is xn--pypal-4ve, a different domain from the genuine ASCII name.

IDNA: which rules decide what is valid

IDNA, short for Internationalizing Domain Names in Applications, defines the conversion and which characters are allowed. Two generations are in use:

Version	Approach	Practical effect
IDNA2003	Case-fold and normalise input, then encode	Maps some characters to others, for example German "ß" to "ss"
IDNA2008	Strict allowed-character tables; no mapping inside the protocol	"ß" is a distinct, valid letter; uppercase and some compatibility characters are rejected as stored labels
UTS #46	Compatibility processing used by browsers	Bridges the two, mapping user input before IDNA2008 checks

The difference is visible in practice. Python's built-in codec implements IDNA2003, so "faß.de".encode("idna") returns b"fass.de", a different name from the one the user typed. The separate third-party idna package follows IDNA2008 and keeps the distinction. Characters that were mapped one way by one tool and another way by a second tool are called deviation characters, and they are worth knowing when you compare names across systems.

How a homograph attack works

Many Cyrillic, Greek and Latin letters share shapes. An attacker registers a name where some or all letters are replaced with such lookalikes, then uses it in a link. If the browser displays the Unicode form, a person sees what looks like a trusted domain. If the browser displays the xn-- form, the trick fails because the string is plainly different.

Three levels of confusion

• Mixed-script labels mix characters from different scripts, such as Latin letters with one Cyrillic letter. These are easiest to detect and block.
• Whole-script confusables use only one script, for example a label made entirely of Cyrillic letters that all resemble Latin ones. No script mixing happens, so mixed-script rules do not catch it.
• Single-script similarity within Latin, such as "ı" (dotless i) or letters with marks, or ASCII-only tricks such as "rn" for "m" and "l" for "I". These are not IDN problems at all but follow the same principle; see typosquatting and lookalike domains.

The practical attack was demonstrated publicly in 2017 using an all-Cyrillic label that looked like a famous brand. Browser vendors then tightened their display rules.

Defences by layer

Layer	Defence
Registry policy	Each TLD publishes which scripts and code points it accepts and often restricts mixing; ICANN guidelines for IDN registration encourage limiting to a language's script and blocking confusable variants. Rules differ widely between registries
Registrar	Some bundle "variants" of a name or hold them to the same registrant; others do little
Browser	Display the Unicode form only when the label passes a policy; otherwise show xn--. Policies generally consider script mixing, specific risky characters and, in some browsers, whether the TLD restricts its registrations
Email and chat clients	Often render links as received; coverage is less consistent than in browsers
Brand owner	Monitor new registrations for visual matches

Browser behaviour has changed over the years and differs between vendors, so avoid assuming that a name which displays as Unicode in one browser does so in another. Test with a current build when it matters.

Using an IDN as a site owner

Registering a name in your own language is legitimate and increasingly common, but it comes with practical trade-offs you should test before launch.

• Certificates. TLS certificates are issued for the A-label. Certificate tooling accepts IDNs, but check that your provider's ordering form handles Unicode input.
• Email. Mail to an IDN domain works when the domain part is punycode. A non-ASCII local part (before the @) needs internationalized email support in every system along the path, which is less universal, so many organisations keep the mailbox name in ASCII.
• Sharing. Links pasted into messages may turn into xn-- strings in some apps, which looks suspicious to recipients. Plan to publish both an ASCII fallback and the IDN.
• Typing. Users need an input method for the script. For many audiences an ASCII transliteration is easier to remember and type; a name in a Latin-script language with diacritics often gets visited without them.
• Variants. If a language has several ways to write the same word (traditional and simplified Chinese, or with and without diacritics), consider registering the relevant variants, or ask the registry whether it bundles them.

Reading an unknown xn-- label quickly

Three checks cover most cases. First, decode it and look at the characters; if the language is one you do not read, paste it into a translator to see whether it is a real word. Second, check scripts: more than one script in a single label is unusual and deserves scrutiny. Third, compare the skeleton against names you care about. If you work with large lists, note that zone files list millions of delegations and only a small portion start with xn--, so a script-based filter is cheap to run on a daily basis.

When you suspect abuse, treat the A-label as the canonical identifier in reports, since it is unambiguous and survives copy and paste across systems. Include the decoded form and the code points (for example U+0430) so the receiving team can see exactly which character is the impostor.

Detecting punycode lookalikes

The input is simple: a list of newly seen names, which in TLD zone files already appear in A-label form (zone files list xn-- names, not the Unicode text). Detection has four steps.

1. Filter. Select labels that start with xn--.
2. Decode. Convert to Unicode and verify the round trip; discard invalid ones.
3. Inspect. Record the scripts in use (Python's unicodedata.name() shows each character's script prefix) and flag mixed scripts or any character that falls in a confusables table.
4. Skeleton match. Map each character to a prototype using the Unicode confusables data (UTS #39) and compare the "skeleton" with your watch list. A label whose skeleton equals paypal is a visual match to that brand.

import unicodedata
label = "xn--pypal-4ve".encode().decode("idna")
scripts = {unicodedata.name(c).split()[0] for c in label}
print(label, scripts)   # pаypal {'LATIN', 'CYRILLIC'}

Remember that a decoded label is not malicious just because it contains non-ASCII characters. Most IDNs are real names in real languages. The signal is the combination of a brand-matching skeleton, a new registration (see newly registered domains) and a mismatch between scripts or between registrant and brand.

Frequently asked questions

What does xn-- mean at the start of a domain label?

It marks a label whose Unicode characters have been encoded in punycode so that DNS can carry it with ASCII letters, digits and hyphens. Software decodes it to show the Unicode form where its display policy allows.

Is every xn-- domain malicious?

No. Most are legitimate names in languages such as Chinese, Japanese, Arabic, Russian, German and Vietnamese. Risk comes from lookalike combinations: labels that mix scripts or whose characters imitate a well-known Latin name.

Why does the same word give different punycode in some tools?

Older IDNA2003 tools map characters such as "ß" to "ss" before encoding, while IDNA2008 keeps "ß" as a distinct letter. Python's built-in codec follows the older rules; the third-party idna package follows IDNA2008.

Do browsers still show homographs in the address bar?

Browsers apply display policies that fall back to the xn-- form for suspicious labels, such as mixed-script ones, and have tightened those rules since public demonstrations in 2017. The details differ by browser and over time, so verify with a current version.

How can I convert a name to punycode?

Use a library or command rather than doing it by hand. In Python, "bücher".encode("idna") returns b"xn--bcher-kva", and .decode("idna") reverses it for a label such as xn--bcher-kva. For IDNA2008 behaviour, use the idna package.

Can an attacker register a mixed-script lookalike in .com?

Registry policy decides. Registries commonly restrict which scripts and characters can be combined, and ICANN guidelines encourage that, but implementations vary by TLD and by script. Check the registry's published IDN table rather than assuming.