{"id":1614,"date":"2021-05-07T16:20:43","date_gmt":"2021-05-07T23:20:43","guid":{"rendered":"https:\/\/coderpad.io\/?p=1614"},"modified":"2024-01-06T14:30:11","modified_gmt":"2024-01-06T22:30:11","slug":"python-list-comprehension-guide","status":"publish","type":"post","link":"https:\/\/coderpad.io\/blog\/development\/python-list-comprehension-guide\/","title":{"rendered":"Python List Comprehension – The Comprehensive Guide"},"content":{"rendered":"\n

Python list comprehensions allow for powerful and readable list mutations. In this article, we’ll learn many different ways in how they can be used and where they’re most useful. <\/p>\n\n\n\n

Python is an incredibly powerful language that\u2019s widely adopted across a wide range of applications. As with any language of sufficient complexity, Python enables multiple ways of doing things. However, the community at large has agreed that code should follow a specific pattern: be \u201cPythonic\u201d. While \u201cPythonic\u201d is a community term, the official language defines what they call \u201cThe Zen of Python\u201d in PEP 20<\/a>. To quote just a small bit of it:<\/p>\n\n\n\n

\n

Explicit is better than implicit.<\/p>\n\n\n\n

Simple is better than complex.<\/p>\n\n\n\n

Complex is better than complicated.<\/p>\n\n\n\n

Flat is better than nested.<\/p>\n<\/blockquote>\n\n\n\n


Introduced in Python 2.0 with PEP 202<\/a>, list comprehensions help align some of these goals for common operations in Python. Let\u2019s explore how we can use list comprehensions and where they serve the Zen of Python better than alternatives.<\/p>\n\n\n\n

What is List Comprehension?<\/h2>\n\n\n\n


Let\u2019s say that we want to make an array of numbers, counting up from 0 to 2. We could assign an empty array, use range<\/code> to create a generator, then append<\/code> to that array using a for<\/code> loop<\/p>\n\n\n

numbers = []\nfor<\/span> x in<\/span> range(3<\/span>):\n    numbers.append(x)<\/code><\/span>Code language:<\/span> Python<\/span> (<\/span>python<\/span>)<\/span><\/small><\/pre>\n\n\n
Alternatively, we could use list comprehension to shorten that to one line of code.<\/p>\n\n\n
numbers = [x for<\/span> x in<\/span> range(3<\/span>)]<\/code><\/span>Code language:<\/span> Python<\/span> (<\/span>python<\/span>)<\/span><\/small><\/pre>\n\n\n
Confused on the syntax? Let\u2019s outline what\u2019s happening token-by-token.<\/p>\n\n\n\n
\"Breakdown<\/figure>\n\n\n\n
The first and last brackets simply indicate that this is a list comprehension. This is also how I remember that a list comprehension outputs an array – it looks like we\u2019re constructing an array with logic inside.<\/p>\n\n\n\n
Second, we have the \u201cx\u201d before the \u201cfor\u201d. This is the return value. This means that if we change the comprehension to:<\/p>\n\n\n
numbers = [x*2<\/span> for<\/span> x in<\/span> range(3<\/span>)]<\/code><\/span>Code language:<\/span> Python<\/span> (<\/span>python<\/span>)<\/span><\/small><\/pre>\n\n\n
Instead of \u201c0, 1, 2\u201d, we\u2019d get \u201c0, 2, 4\u201d.<\/p>\n\n\n\n
Next up, we have a declaration of a for<\/code> loop. This comprises of three separate parts:<\/p>\n\n\n\n
    \n
  1. \u201cfor\u201d – the start of the loop<\/li>\n\n\n\n
  2. \u201cx\u201d – declaring the name of the variable to assign in each iteration<\/li>\n\n\n\n
  3. \u201cin\u201d – denoting the start of listening for the iterator<\/li>\n<\/ol>\n\n\n\n
    Finally, we have the range<\/code>. This acts as the iterator for the for<\/code> loop to iterate through. This can be replaced with anything a for<\/code> loop can go through: a list, a tuple, or anything else that implements the iterator interface.<\/p>\n\n\n\n
    Are List Comprehension Pythonic?<\/h2>\n\n\n\n
    While it might seem counterintuitive to learn a new syntax for manipulating lists, let\u2019s look at what the alternative looks like. Using map<\/code>, we can pass an anonymous function (lambda) to multiply a number by 2, pass the range<\/code> to iterate through. However, once this is done, we\u2019re left with a map<\/code> object. In order to convert this back to a list, we have to wrap that method in list<\/code>.<\/p>\n\n\n
    numbers = list(map(lambda<\/span> x: x*2<\/span>, range(3<\/span>)))<\/code><\/span>Code language:<\/span> Python<\/span> (<\/span>python<\/span>)<\/span><\/small><\/pre>\n\n\n
    Compare this to the list comprehension version:<\/p>\n\n\n
    numbers = [x*2<\/span> for<\/span> x in<\/span> range(3<\/span>)]<\/code><\/span>Code language:<\/span> Python<\/span> (<\/span>python<\/span>)<\/span><\/small><\/pre>\n\n\n
    Looking at the comprehension, it\u2019s significantly more readable at a glance. Thinking back to The Zen of Python, \u201cSimple is better than complex,\u201d list comprehensions seem to be more Pythonic than using map<\/code>.<\/p>\n\n\n\n
    While others might argue that a \u201cfor\u201d loop might be easier to read, the Zen of Python also mentions \u201cFlat is better than nested\u201d. Because of this, list comprehensions for simple usage like this are more Pythonic.<\/p>\n\n\n\n
    Now that we\u2019re more familiar with basic usage of list comprehension, let\u2019s dive into some of it\u2019s more powerful capabilities.<\/p>\n\n\n\n
    Filtering<\/h2>\n\n\n\n
    While it might seem like list comprehension is only capable of doing a 1:1 match like map<\/code>, you\u2019re actually able to implement logic more similar to filter<\/code> to change how many items are in the output compared to what was input.<\/p>\n\n\n\n
    If we add an if<\/code> to the end of the statement, we can limit the output to only even numbers:<\/p>\n\n\n
    even_numbers = [x for<\/span> x in<\/span> range(10<\/span>) if<\/span> x%2<\/span>==0<\/span>] #[0, 2, 4, 6, 8]<\/span><\/code><\/span>Code language:<\/span> Python<\/span> (<\/span>python<\/span>)<\/span><\/small><\/pre>\n\n\n
    This can of course be combined with the changed mutation value:<\/p>\n\n\n
    double_even_numbers = [x*2<\/span> for<\/span> x in<\/span> range(10<\/span>) if<\/span> x%2<\/span>==0<\/span>] #[0, 4, 8, 12, 16]<\/span><\/code><\/span>Code language:<\/span> Python<\/span> (<\/span>python<\/span>)<\/span><\/small><\/pre>\n\n\n
    Conditionals<\/h2>\n\n\n\n
    While filtering might seem like the only usage of if<\/code> in a list comprehension, you\u2019re able to use them to act as conditionals to return different values from the original.<\/p>\n\n\n
    number_even_odd = [\"Even\"<\/span> if<\/span> x % 2<\/span> == 0<\/span> else<\/span> \"Odd\"<\/span> for<\/span> x in<\/span> range(4<\/span>)]\n# [\"Even\", \"Odd\", \"Even\", \"Odd\"]<\/span><\/code><\/span>Code language:<\/span> Python<\/span> (<\/span>python<\/span>)<\/span><\/small><\/pre>\n\n\n
    Keep in mind, you could even combine this ternary method with the previous filtering if<\/code>:
    <\/p>\n\n\n
    thirds_even_odd = [\"Even\"<\/span> if<\/span> x % 2<\/span> == 0<\/span> else<\/span> \"Odd\"<\/span> for<\/span> x in<\/span> range(10<\/span>) if<\/span> x%3<\/span>==0<\/span>]\n# [0, 3, 6, 9] after filtering numbers<\/span>\n# [\"Even\", \"Odd\", \"Even\", \"Odd\"] after ternary to string<\/span><\/code><\/span>Code language:<\/span> Python<\/span> (<\/span>python<\/span>)<\/span><\/small><\/pre>\n\n\n
    If we wanted to expand this code to use full-bodied functions, it would look something like this:<\/p>\n\n\n
    thirds_even_odd = []\nfor<\/span> x in<\/span> range(10<\/span>):\n    if<\/span> x%3<\/span>==0<\/span>:\n        if<\/span> x%2<\/span>==0<\/span>:\n            thirds_even_odd.append(\"Even\"<\/span>)\n        else<\/span>:\n            thirds_even_odd.append(\"Odd\"<\/span>)<\/code><\/span>Code language:<\/span> Python<\/span> (<\/span>python<\/span>)<\/span><\/small><\/pre>\n\n\n
    Nested Loops<\/h2>\n\n\n\n
    While we explained that you\u2019re able to have less items in the output than the input in our \u201cfiltering\u201d section, you\u2019re able to do the opposite as well. Here, we\u2019re able to nest two \u201cfor\u201d loops on top of each other in order to have a longer output than our initial input.<\/p>\n\n\n
    repeated_list = [y for<\/span> x in<\/span> [\"\"<\/span>, \"\"<\/span>] for<\/span> y in<\/span> [1<\/span>, 2<\/span>, 3<\/span>]]\n# [1, 2, 3, 1, 2, 3]<\/span><\/code><\/span>Code language:<\/span> Python<\/span> (<\/span>python<\/span>)<\/span><\/small><\/pre>\n\n\n
    This logic allows you to iterate through two different arrays and output the final value in the nested loop. If we have to rewrite this, we\u2019d write it out as:<\/p>\n\n\n
    repeated_list = []\nfor<\/span> x in<\/span> [\"\"<\/span>, \"\"<\/span>]:\n    for<\/span> y in<\/span> [1<\/span>, 2<\/span>, 3<\/span>]:\n        repeated_list.append(y)<\/code><\/span>Code language:<\/span> Python<\/span> (<\/span>python<\/span>)<\/span><\/small><\/pre>\n\n\n
    This allows us to nest the loops and keep the logic flat. However, you\u2019ll notice in this example, we\u2019re not utilizing the \u201cx\u201d variable. Let\u2019s change that and do a calculations based on the \u201cx\u201d variable as well:<\/p>\n\n\n
    numbers_doubled = [y for<\/span> x in<\/span> [1<\/span>, 2<\/span>] for<\/span> y in<\/span> [x, x*2<\/span>]]\n# 1, 2, 2, 4<\/span><\/code><\/span>Code language:<\/span> Python<\/span> (<\/span>python<\/span>)<\/span><\/small><\/pre>\n\n\n
    Now that we\u2019ve explored using hard-coded arrays to nest loops, let\u2019s go one level deeper and see how we can utilize list comprehensions in a nested manner.<\/p>\n\n\n\n
    Nested Comprehensions<\/h2>\n\n\n\n
    There are two facts that we can combine to provide list comprehension with a super power:<\/p>\n\n\n\n
      \n
    1. You can use lists inside of a list comprehension<\/li>\n\n\n\n
    2. List comprehensions returns lists<\/li>\n<\/ol>\n\n\n\n
      Combining these leads to the natural conclusion that you can nest list comprehensions inside of other list comprehensions.<\/p>\n\n\n\n
      For example, let\u2019s take the following logic that, given a two-dimensional list, returns all of the first index items in one list and the second indexed items in a second list.<\/p>\n\n\n
      row_list = [[1<\/span>, 2<\/span>], [3<\/span>,4<\/span>], [5<\/span>,6<\/span>]]\nindexed_list = []\nfor<\/span> i in<\/span> range(2<\/span>):\n    indexed_row = []\n    for<\/span> row in<\/span> row_list:\n        indexed_row.append(row[i])\n    indexed_list.append(indexed_row)\n\nprint(indexed_list)\n# [[1, 3, 5], [2, 4, 6]]<\/span><\/code><\/span>Code language:<\/span> Python<\/span> (<\/span>python<\/span>)<\/span><\/small><\/pre>\n\n\n
      You\u2019ll notice that the first indexed items (1<\/code>, 3<\/code>, 5<\/code>) are in the first array, and the second indexed items (2<\/code>, 4<\/code>, 6<\/code>) are in the second array.<\/p>\n\n\n\n
      Let\u2019s take that and convert it to a list comprehension:<\/p>\n\n\n
      row_list = [[1<\/span>, 2<\/span>], [3<\/span>,4<\/span>], [5<\/span>,6<\/span>]]\nindexed_list = [[row[i] for<\/span> row in<\/span> row_list] for<\/span> i in<\/span> range(2<\/span>)]\nprint(indexed_list)\n# [[1, 3, 5], [2, 4, 6]]<\/span><\/code><\/span>Code language:<\/span> Python<\/span> (<\/span>python<\/span>)<\/span><\/small><\/pre>\n\n\n
      Readable Actions and Other Operators<\/h2>\n\n\n\n
      Something you may have noticed while working with list comprehension is how close some of these operators are to a typical sentence. While basic comprehensions serve this well on their own, they\u2019re advanced by the likes of Python\u2019s other grammatical-style operators. For example, operators may include:<\/p>\n\n\n\n
        \n
      • and<\/code> – Logical \u201cand\u201d<\/li>\n\n\n\n
      • or<\/code> – Logical \u201cor\u201d<\/li>\n\n\n\n
      • not<\/code> – Logical \u201cnot\u201d<\/li>\n\n\n\n
      • is<\/code> – Equality check<\/li>\n\n\n\n
      • in<\/code> – Membership check\/second half of for<\/code> loop<\/li>\n<\/ul>\n\n\n\n
        These can be used for great effect. Let\u2019s look at some options we could utilize:<\/p>\n\n\n
        vowels = 'aeiou'<\/span>\nword = \"Hello!\"<\/span>\n\nword_vowels = [letter for<\/span> letter in<\/span> word if<\/span> letter.lower() in<\/span> vowels]\n\nprint(word_vowels)\n\n# ['e', 'o']<\/span><\/code><\/span>Code language:<\/span> Python<\/span> (<\/span>python<\/span>)<\/span><\/small><\/pre>\n\n\n
        Alternatively we could check for consonants instead, simply by adding one \u201cnot\u201d:<\/p>\n\n\n
        word_consonants = [letter for<\/span> letter in<\/span> word if<\/span> letter.lower() not<\/span> in<\/span> vowels]\n# ['H', 'l', 'l']<\/span><\/code><\/span>Code language:<\/span> Python<\/span> (<\/span>python<\/span>)<\/span><\/small><\/pre>\n\n\n
        Finally, to showcase boolean logic, we\u2019ll do a slight contrived check for numbers that mod 2 and 3 perfectly but are not 4:
        <\/p>\n\n\n
        restricted_number = 4<\/span>\n\nsafe_numbers = [x for<\/span> x in<\/span> range(6<\/span>) if<\/span> (x%2<\/span>==0<\/span> or<\/span> x%3<\/span>==0<\/span>) and<\/span> x is<\/span> not<\/span> restricted_number]\n\n# [0, 2, 3]<\/span><\/code><\/span>Code language:<\/span> Python<\/span> (<\/span>python<\/span>)<\/span><\/small><\/pre>\n\n\n
        Conclusion & Challenge<\/h2>\n\n\n\n

        We\u2019ve covered a lot about list comprehension in Python today! We\u2019re able to build complex logic into our applications while maintaining readability in most situations. However, like any tool, list comprehension can be abused. When you start including too many logical operations to comfortably read, you should likely migrate away from list comprehension to use full-bodied for<\/code> loops.

        For example, given this sandbox code pad of a long and messy list comprehension, how can you refactor to remove all usage of list comprehensions? Avoid using map<\/code>, filter<\/code> or other list helpers, either. Simply use nested for<\/code> loops and if<\/code> conditionals to match the behavior as it was before.<\/p>\n\n\n\n\t