Usage

Usage¶

Six expressions, all taking a string column of URLs or bare hostnames:

	`https://www.bbc.co.uk/news`
`tld.extract`	`{"www", "bbc", "co.uk", false}`	struct: `subdomain`, `domain`, `suffix`, `is_private`
`tld.fqdn`	`www.bbc.co.uk`	the whole hostname
`tld.registrable_domain`	`bbc.co.uk`	what you register — `domain.suffix`
`tld.subdomain`	`www`
`tld.domain`	`bbc`	the registrable label, `tldextract`'s `domain` field
`tld.suffix`	`co.uk`	the public suffix

Nulls, and the one exception¶

The five single-value expressions return null for a part that does not exist. That matters in a DataFrame: an empty string is a value, so two rows that both failed to parse would compare equal and join to each other.

tld.extract is the exception, and deliberately so — it reproduces tldextract.ExtractResult verbatim, empty strings and all. Reach for it when porting existing tldextract code and you want the behavior unchanged; reach for anything else when the result is going into a join, a group-by, or a comparison.

`fqdn` vs. `registrable_domain`¶

The two differ on hosts that have no registrable domain. registrable_domain is strict — no recognized suffix means null, so an IP or a .local name drops out. fqdn just gives you the hostname:

tld.fqdn("url")  # "127.0.0.1", "localhost", "printer.local"
tld.registrable_domain("url")  # null,        null,        null

fqdn is also the normalized netloc — scheme, userinfo, port, path, query and fragment stripped, trailing root labels dropped, and the non-ASCII IDNA separators folded to . — so ftp://user:pw@ftp.gnu.org:2121/pub becomes ftp.gnu.org. Casing and punycode spelling are preserved, exactly like tldextract.

Expression namespace¶

Importing the package registers a .tld namespace:

df.with_columns(pl.col("url").tld.registrable_domain())
df.filter(pl.col("url").tld.suffix() == "org")

Scalars¶

For code that isn't holding a DataFrame — the same Rust core, no Polars round-trip:

tld.extract_scalar("https://www.bbc.co.uk/news")
# ('bbc.co.uk', 'bbc', 'co.uk')         (registrable_domain, domain, suffix), nulls for absences

tld.extract_scalar_full("https://www.bbc.co.uk/news")
# ('www', 'bbc', 'co.uk', False)        (subdomain, domain, suffix, is_private), tldextract-faithful

Private suffixes¶

The Public Suffix List has an ICANN section and a private section. Like tldextract, the private section is off by default:

tld.extract_scalar("pola-rs.github.io")
# ('github.io', 'github', 'io')

tld.extract_scalar("pola-rs.github.io", include_private=True)
# ('pola-rs.github.io', 'pola-rs', 'github.io')

Every expression takes the same include_private keyword.

Migrating from 0.1¶

parts() and top_domain() still work in 0.2 but emit a DeprecationWarning, and are removed in 0.3.

0.1	0.2
`tld.parts(...).struct.field("full_domain")`	`tld.registrable_domain(...)`
`tld.parts(...).struct.field("sld")`	`tld.domain(...)`
`tld.parts(...).struct.field("tld")`	`tld.suffix(...)`
`tld.top_domain(...)`	`tld.domain(...)`

Two things to watch when you migrate:

full_domain was a misnomer. It held the registrable domain (bbc.co.uk). The actual full domain is fqdn() (www.bbc.co.uk), which is new in 0.2.
top_domain() returned "" where there was no registrable label; domain() returns null. If you were comparing against "", compare against null instead — or read the field off extract(), which keeps the empty-string form.