PHP: What to do about "Allowed memory size exhausted"?
Fatal error: Allowed memory size of 8388608 bytes exhausted
(tried to allocate 12288 bytes)
The first answer you find for this is: raise the limit. Sometimes that is even right. Usually, though, it is a reaction to a file being loaded into memory in one piece when it could have been read line by line — and the difference is a factor of a thousand.
First of all: where do I stand?
Before changing anything, it is worth looking at the numbers:
<?php
echo ini_get('memory_limit'), "\n";
echo memory_get_usage(), "\n";
echo memory_get_usage(true), "\n";
echo memory_get_peak_usage(true), "\n";
?>
memory_limit : 512M
memory_get_usage() : 0.46 MiB
memory_get_usage(true) : 2.00 MiB
The difference between the two needs explaining. Without a parameter you get
what PHP is using for your variables. With true you get what PHP has
requested from the operating system — and that is the figure the limit is
measured against. So for the question “how close am I” true is the more
honest one.
memory_get_peak_usage() stays at the high-water mark, even once the memory
has long been freed:
peak so far : 6.28 MiB
current : 0.46 MiB
That is the value you want to write to your log at the end of a script. Because when your script crashes, it was the peak that broke the limit — not the usage at the end.
The one point that usually decides it
The same file, 11.63 MiB, 200,000 lines. Three ways to read it:
file_get_contents 11.64 MiB
file() as an array 22.31 MiB
fgets() line by line 0.01 MiB
The third line is not a typo. Ten kilobytes.
file_get_contents() puts the whole file down as one string, so about the file
size as expected. file() needs nearly double, because each of the 200,000
lines becomes its own string with its own bookkeeping. And the streaming route
needs practically nothing, because only a single line is in memory at any one
time.
<?php
$fh = fopen($file, 'r');
while (($line = fgets($fh)) !== false) {
// process one line
}
fclose($fh);
?>
With a 12 MiB file that still makes no difference. With a 2 GiB log file it
decides whether your script runs at all — and there no memory_limit = -1
helps either, because the server simply does not have the memory.
The same principle applies elsewhere:
- Database queries: not
fetchAll(), butfetch()in a loop. - CSV:
fgetcsv()rather than reading the file and splitting it yourself. - JSON: a streaming parser for large documents, because
json_decode()builds the complete document in memory. - Images: GD keeps an image uncompressed in memory, four bytes per pixel. A 4000×3000 photo is maybe 2 MiB as a JPEG — measured, it occupies 46.91 MiB in GD, matching the calculation 4000 × 3000 × 4 bytes. With a default limit of 128 MiB, three such images at once are already too many.
What an array really costs
So that the numbers are less of a surprise:
range(1, 100000) : 2.00 MiB -> 21.0 bytes per integer
100,000 short strings : 5.82 MiB
An integer on its own occupies 8 bytes. In an array it costs 21, because PHP maintains hash table entries and type information for every element.
That explains why 50,000 database rows with ten columns each need so much more memory than the sum of the field contents would suggest.
Copy on write
An effect that makes tracking down the cause harder:
$copy = $big : 0.00 MiB <- no copy yet
after one modification : 4.00 MiB <- now it was copied
The assignment costs nothing. PHP merely notes that two variables point at the same data. Only when you modify one of them does the copy actually happen.
For debugging that means: memory does not blow up on the line with the assignment, but somewhere further down, on a harmless-looking modification. Anyone who does not know this looks in the wrong place.
Catching the error — you cannot, but you can read it
The obvious attempt:
<?php
try {
// use a lot of memory
} catch (Throwable $e) {
echo 'caught'; // never reached
}
?>
Does not work. Exceeding the limit is a fatal error and not an Error object,
so there is nothing to catch. The catch branch is not entered, the script
simply ends at that point.
What does work:
<?php
register_shutdown_function(function (): void {
$f = error_get_last();
if ($f !== null && $f['type'] === E_ERROR) {
// log here, clean up, serve an error page
error_log($f['message'] . ' in ' . $f['file'] . ':' . $f['line']);
}
});
?>
[shutdown] Allowed memory size of 8388608 bytes exhausted (tried to allocate 12288 bytes)
[shutdown] in sprengen.php line 16
The function still runs after a fatal error. So you do get the file and the line — and that is exactly the information you need. You just cannot let the script carry on.
The process’s exit code, incidentally, is 255, the same as for any other fatal error.
And when do you raise the limit after all?
There are legitimate cases. An import script that runs once a month and really does have to hold a large data set in memory may have more than a web page:
<?php
ini_set('memory_limit', '512M');
?>
before : 512M -> after: 256M -> again : 64M
That works at runtime, in both directions. Two caveats: it does not help with a parse error, because the line is never executed (more on that in the article about error messages), and some hosts lock the directive down.
What I do not recommend is memory_limit = -1 on a web server. The limit is
not there to annoy you, it is there so that a single mistake — an infinite
loop, a query without a LIMIT — does not take the whole server with it.
Without a limit you get, instead of a fatal error in one process, a server that
starts swapping or gets killed by the OOM killer.
For a command line script that you start yourself, -1 is perfectly fine.
Summary
- Measure first: log
memory_get_peak_usage(true)at the end of the script. - Read files line by line rather than in one piece — 0.01 MiB instead of 11.64 MiB in the test.
file()needs double the file size.- An integer in an array costs 21 bytes, not 8.
- Copy on write: memory is claimed on modification, not on assignment.
- The error cannot be caught, but can be read via
register_shutdown_function()anderror_get_last(). memory_limit = -1belongs on the command line, not on a web server.
About Netcup (advertisement)
The German host Netcup offers, among other things, affordable and powerful web hosting packages, KVM-based root servers and dedicated servers. With our voucher codes you can save even more (6€ off your first order, 30% off all KVM-based root servers, ...).