PHP: How do you parse HTML properly?

You want to pull one piece of information out of an HTML page, and you reach for the first tool that comes to mind: a regular expression. That usually goes wrong. PHP ships with everything you need for this job.

All output comes from a script I ran against PHP 8.5.9 with libxml 2.9.14.

The tool: DOMDocument

You do not need to install anything. The dom, libxml and simplexml extensions are part of PHP. For HTML you use DOMDocument:

<?php
  $dom = new DOMDocument();
  $dom->loadHTML($html);
?>

The part many people do not know: loadHTML() copes with broken HTML. Behind it sits libxml’s HTML parser, which is built for exactly that. Take this deliberately malformed markup — an unclosed <p>, an attribute without quotes, uppercase tags, a bare &:

<html><body>
  <DIV class=teaser>
    <p>First product & accessories
    <a href="/a.html">Product A</a>
    <p>Second product
    <a href='/b.html'>Product B</a>
  </DIV>
</body></html>

That loads without trouble.

Do not forget libxml_use_internal_errors

With broken HTML, libxml writes warnings to the log or the output. You usually do not want that. libxml_use_internal_errors(true) collects the messages instead of letting them be printed:

<?php
  $dom = new DOMDocument();
  libxml_use_internal_errors(true);
  $dom->loadHTML($html);

  $errors = libxml_get_errors();   // in case you do want to look at them
  libxml_clear_errors();
?>

For our example above libxml reports exactly one problem (htmlParseEntityRef: no name, because of the bare &) — and parses on regardless.

Extracting data with XPath

For searching you use DOMXPath. With it you describe what you want, instead of describing what the characters around it look like:

<?php
  $xpath = new DOMXPath($dom);

  foreach ($xpath->query('//div[@class="teaser"]//a') as $a) {
    echo $a->textContent . ' -> ' . $a->getAttribute('href') . "\n";
  }
?>

Output:

Product A    -> /a.html
Product B    -> /b.html

You read attributes with getAttribute() and the text with textContent. A practical example, summing prices from a list:

RS 1000    10.40 EUR
CCX11      23.68 EUR
sum: 34.08 EUR

If you need the markup of a subtree rather than its text, use saveHTML() with a node as its argument:

<?php
  echo $dom->saveHTML($node);
?>

The character encoding trap

And now the bug that can cost you an afternoon. Without a hint about the encoding, loadHTML() assumes ISO-8859-1. With UTF-8 input you get this:

without charset   : Preis in Euro: 10,40 ⬠für Zubehör

There are two ways to fix it. Either the document contains a <meta charset="utf-8">, in which case libxml detects the encoding itself, or you prepend an XML declaration to the markup:

<?php
  $dom->loadHTML('<?xml encoding="UTF-8">' . $html);
?>

Both give the correct result:

with XML hint     : Preis in Euro: 10,40 € für Zubehör
with meta charset : Preis in Euro: 10,40 € für Zubehör

So if your accented characters are being mangled, this is why.

SimpleXML is not an HTML parser

SimpleXML is convenient, but it wants well-formed XML. Pointed at our broken HTML:

simplexml_load_string($html)   => false

For XML — an RSS feed, say — SimpleXML is a good choice. For HTML from the wild it is not.

Why not just use a regular expression?

Because it falls over at the first gust of wind. Take a perfectly reasonable looking expression for finding links:

<?php
  preg_match_all('#<a href="([^"]+)">([^<]+)</a>#', $html, $matches);
?>

The result with our example:

regular expression : 1 of 2 links
DOM                : 2 of 2 links

The second link uses single instead of double quotes. A single character of difference in the markup, and the expression misses half the results. That is exactly the point: an HTML parser already knows the rules of HTML. A regular expression has to be taught them again every time — and you will always forget a case.

That does not mean regular expressions are forbidden. For very simple extractions from markup whose shape you know, they are fast and perfectly fine. But as soon as nesting, attributes in arbitrary order or somebody else’s markup are involved, use a parser.

What about all those libraries?

On Stack Overflow you will find a long list of libraries for this job. That list has aged, though: several of the projects named there are no longer maintained, others are in security-only mode. And the two best known ones that are not built on libxml are explicitly not recommended in that same answer, because they are slow and memory hungry.

My advice: start with DOMDocument and DOMXPath. It is built in, it is fast, and you can do everything with it. If you would rather write CSS selectors than XPath, the DomCrawler from the Symfony components is a good and actively maintained addition. It can be used without the rest of Symfony.

Summary

  • DOMDocument::loadHTML() handles broken HTML — that is not a myth.
  • libxml_use_internal_errors(true) turns off the shower of warnings.
  • Use DOMXPath for searching.
  • Without <meta charset> or <?xml encoding="UTF-8"> your accented characters get mangled.
  • SimpleXML is for XML, not for HTML.
  • Regular expressions break as soon as the markup changes even slightly.

About Netcup (advertisement)

The German host Netcup offers, among other things, affordable and powerful web hosting packages, KVM-based root servers and dedicated servers. With our voucher codes you can save even more (6€ off your first order, 30% off all KVM-based root servers, ...).