How to merge two dictionaries in Python
Merging two dictionaries used to take two statements. Since Python 3.9 it takes one character. This article shows the options, and the one property of all of them that causes trouble in real code.
The short answer
merged = a | b
a = {'host': 'localhost', 'port': 3306}
b = {'port': 5432, 'user': 'admin'}
a | b : {'host': 'localhost', 'port': 5432, 'user': 'admin'}
a is unchanged : {'host': 'localhost', 'port': 3306}
The | operator for dicts arrived in Python 3.9 with
PEP 584. On Python 3.5 to 3.8 the
equivalent is dictionary unpacking, from
PEP 448:
merged = {**a, **b}
{**a, **b} : {'host': 'localhost', 'port': 5432, 'user': 'admin'}
Both build a new dict and leave the originals alone.
The right-hand side wins
a | b -> port = 5432 (b's value)
b | a -> port = 3306 (a's value)
Which is what you want for the common case of defaults on the left and overrides on the right:
config = DEFAULTS | user_config
Ordering is slightly more subtle than the value rule. A key that appears in both keeps the position of its first appearance but takes the value of its last:
x = {'one': 1, 'two': 2}
y = {'two': 22, 'zero': 0}
x | y = {'one': 1, 'two': 22, 'zero': 0}
two stayed in second place and took 22.
Merging in place
If you want to modify the left-hand dict rather than build a new one:
a |= b # since 3.9
a.update(b) # any version
Both do the same thing. Which brings us to the mistake everybody makes once:
update() returns : None
dict.update()
returns None, like most methods that mutate in place. So
merged = a.update(b) # merged is None
does not merge anything into merged. It mutates a and hands you None. If
you are lucky the next line raises AttributeError on NoneType; if you are
not, the None travels a while first.
Three or more
| chains, and so does unpacking:
e | f | g : {'a': 1, 'b': 2, 'c': 3}
{**e, **f, **g} : {'a': 1, 'b': 2, 'c': 3}
If the dicts are in a list, loop:
merged = {}
for one in dicts:
merged |= one
from a list, in a loop : {'a': 1, 'b': 2, 'c': 3}
Now the part that bites: the merge is shallow
Every form above copies one level deep. Watch what happens to a nested config:
left = {'db': {'host': 'localhost', 'port': 3306}, 'debug': False}
right = {'db': {'port': 5432}}
left | right = {'db': {'port': 5432}, 'debug': False}
host is gone. The db key existed on both sides, so the whole value from the
right replaced the whole value from the left. Nothing looked inside them.
This is not a corner case. Nested dicts are exactly what configuration looks like, and “I overrode one setting and lost the rest of the section” is the result.
If you want the sections merged too, you have to write it:
def deep_merge(lhs: dict, rhs: dict) -> dict:
out = dict(lhs)
for key, value in rhs.items():
if isinstance(out.get(key), dict) and isinstance(value, dict):
out[key] = deep_merge(out[key], value)
else:
out[key] = value
return out
deep_merge : {'db': {'host': 'localhost', 'port': 5432}, 'debug': False}
Shallow also means the halves are shared
The second half of the same problem, and the sneakier one:
base = {"cfg": {"n": 1}}
copy = base | {}
copy["cfg"]["n"] = 99
after changing the copy, base = {'cfg': {'n': 99}}
same inner object? True
The outer dict is new. The inner one is the same object, referenced from both. Writing through what you thought was a copy changed the original.
Only copy.deepcopy
gives you an independent structure:
after copy.deepcopy and changing the copy, base = {'cfg': {'n': 1}}
same inner object? False
ChainMap, when you do not want to merge at all
collections.ChainMap
searches several dicts in order without building anything:
ChainMap(b, a)['port'] : 5432 <- first mapping wins, so b
Two things to note. The precedence is reversed compared to | — here the
first mapping wins, not the last. And because nothing was copied, it sees later
changes to the underlying dicts:
after changing a, cm['host'] : changed-later
That is either exactly what you want or a nasty surprise, depending on whether you knew about it. It suits layered configuration well: command line over environment over file over defaults, with no copying and no ambiguity about where a value came from.
Two errors you may run into
{**a, **b} handles any hashable key. dict(**a, **b), which you will see in
older answers, does not:
n1 | n2 : {1: 'one', 2: 'two'}
{**n1, **n2} : {1: 'one', 2: 'two'}
dict(**n1, **n2) -> TypeError: keywords must be strings
And | insists on two dicts, where update() is happy with any iterable of
pairs:
dict | list -> TypeError: unsupported operand type(s) for |: 'dict' and 'list'
d.update(pairs) works : {'a': 1, 'b': 2}
About Netcup (advertisement)
The German host Netcup offers, among other things, affordable and powerful web hosting packages, KVM-based root servers and dedicated servers. With our voucher codes you can save even more (6€ off your first order, 30% off all KVM-based root servers, ...).