Filtering Dictionary In Python 3
“RuntimeError: Dictionary changed size during iteration”, Python slaps you in your face when you try to add/remove entries in
dict object during iteration.
In this post, I will walk us through how we can filter a
dict collection. First, let us try to understand the problem by running this snippet in your interpreter:
Try to run this snippet!
You will get a RuntimeError, as expected.
“RuntimeError: dictionary changed size during iteration”, Python interpreter
The error message is self-explanatory and this makes perfect sense, you shouldn’t be able to expand/shrink the collection while you are looping through it. Imagine if you are allowed to do so, and you keep expanding the collection in your loop body:
If the above snippet is valid in Python, we will slump into infinite loop until memory is run out.
So, in any event we want to filter a
dict object in Python, can we circumvent away from the RuntimeError? Of course!
Solution #1: Use A Shallow Copy
This is the least favorable solution, given that we have to create a new copy of the dictionary.
The above code will work, but it’s less hygienic to me.
Solution #2: Use A Copy Of Dictionary Keys
Another solution is instead, create a copy of dictionary keys as our iterator:
Solution #3: Deferred Update
If you are against of create any copy of either the source dictionary or the keys, we can also use the “deferred update” strategy. Deferred update, as the name tells, is to defer any mutation to the dictionary until the iteration is over.
However, a caveat here. Imagine an uncaught exception in between line 9 and 10, the iteration will break, all pending updates will be void and your
dict object will be left intact.
I personally in favor of using copy of dictionary keys anddeferred update. In the subsequent part of this post, I will use the former to illustrate how we can make our filter logic reusable.
Making The Logic Reusable
At this point, I decided to use a copy of dictionary keys to mitigate the problem when attempting to delete entries during iteration. Let’s see how we can make the logic reusable.
In my case, I created
FilterableDict — a subclass of the built-in
dict class to add
FilterableDict, subclass of built-in dict class
To further reduce the memory footprint of our key copy, I use a
tuple instead of a
list collection to store them.
Let’s make use of our
FilterableDict and observe the output:
Using our FilterableDict
Here we go, a reusable class to support filtering for a dictionary in Python.
Full Source Code:
A different way to achieve the same purpose? Leave me a comment :)
First published on 2019-07-01
Republished on Hackernoon