{"id":19533,"date":"2022-09-28T04:47:40","date_gmt":"2022-09-28T11:47:40","guid":{"rendered":"https:\/\/coderpad.io\/?p=19533"},"modified":"2023-06-07T13:58:36","modified_gmt":"2023-06-07T20:58:36","slug":"window-functions-aggregate-data-postgres","status":"publish","type":"post","link":"https:\/\/coderpad.io\/blog\/development\/window-functions-aggregate-data-postgres\/","title":{"rendered":"Why & How to Use Window Functions to Aggregate Data in Postgres"},"content":{"rendered":"\n

A window function is a feature developed in PostgreSQL — available since version 8.4<\/a>— to analyze data beyond the current row (hence the term “window”).<\/p>\n\n\n\n

These windows can aggregate information to each row of your output. Apart from aggregate functions<\/a> and groupings, window functions provide another way to perform calculations based on the values of several records.<\/p>\n\n\n\n

Using window functions removes the hassle of using subqueries and <\/strong>JOINS<\/code> to aggregate neighboring rows that can be related to the current row.<\/strong><\/p>\n\n\n\n

This tutorial will show you how aggregate functions are used as window functions. You will learn how to use MAX()<\/code>, MIN()<\/code>, AVG()<\/code>, SUM()<\/code>, ROW_NUMBER()<\/code>, RANK()<\/code>, and DENSE_RANK()<\/code> as window functions. But first, you need to know why you should use them.<\/p>\n\n\n\n

\n

\u2705 You can use the CoderPad sandbox at the bottom of this page<\/a> or as a new browser window<\/a> to run the queries in this tutorial — the table is already loaded for you!<\/p>\n<\/blockquote>\n\n\n\n

Why use window functions<\/strong><\/h2>\n\n\n\n

To get an idea of why you should use window functions, let\u2019s start with an example database:<\/p>\n\n\n

\n
\"\"\/<\/a><\/figure>\n<\/div>\n\n\n

Using this data, we want to get a breakdown of salaries for each employee in every department. We want to analyze the max salary across each department to compare it with each employee record.<\/p>\n\n\n\n

To get that query result, you would probably use a Common Table Expression (CTE)<\/a> with a JOIN<\/code> statement like the following:<\/p>\n\n\n

WITH<\/span> dep_stats AS<\/span> (\n\u00a0 \u00a0 SELECT<\/span> dep_name, max(salary) max_salary\n\u00a0 \u00a0 FROM<\/span> emp_salaries\n\u00a0 \u00a0 GROUP<\/span> BY<\/span> dep_name\n)\n\nSELECT<\/span> e.*, d.max_salary\nFROM<\/span> emp_salaries e\nLEFT OUTER JOIN<\/span> dep_stats d ON<\/span> d.dep_name = e.dep_name;<\/code><\/span>Code language:<\/span> PostgreSQL SQL dialect and PL\/pgSQL<\/span> (<\/span>pgsql<\/span>)<\/span><\/small><\/pre>\n\n\n
That query will give you this output:<\/p>\n\n\n
\n
\"\"\/<\/a><\/figure>\n<\/div>\n\n\n
But a better way to get the same result is to use a window function like the following:<\/p>\n\n\n
SELECT<\/span> e.*,\n     MAX(salary) OVER<\/span>(PARTITION<\/span> BY<\/span> dep_name) max_salary\nFROM<\/span> emp_salaries e;<\/code><\/span>Code language:<\/span> PostgreSQL SQL dialect and PL\/pgSQL<\/span> (<\/span>pgsql<\/span>)<\/span><\/small><\/pre>\n\n\n
As you can see, using a window function makes writing complex SQL queries easy. It allows you to output the current rows and add aggregate values accordingly.<\/p>\n\n\n\n
\n
\u2139\ufe0f Notice that window functions allowed us to perform aggregation without using the GROUP BY<\/code> clause, as in the case of aggregate functions.<\/p>\n<\/blockquote>\n\n\n\n
Create a sample table<\/strong><\/h2>\n\n\n\n
In this tutorial, you can experiment with your own table or create a sample table that we will borrow from the PostgreSQL documentation<\/a> with a simple tweak. If you’re doing this tutorial from your own IDE, you can create your sample table by running the following script:<\/p>\n\n\n
DROP<\/span> TABLE<\/span> IF<\/span> EXISTS<\/span> emp_salaries;\n\nCREATE<\/span> TABLE<\/span> emp_salaries (\n  emp_id SERIAL<\/span> PRIMARY KEY<\/span>,\n  emp_name VARCHAR<\/span>(30<\/span>),\n  dep_name VARCHAR<\/span>(40<\/span>),\n  salary DECIMAL<\/span>\n);\n\nINSERT<\/span> INTO<\/span> emp_salaries VALUES<\/span> (11<\/span>, 'Ahmed'<\/span>, 'Engineering'<\/span>, 5200<\/span>);\nINSERT<\/span> INTO<\/span> emp_salaries VALUES<\/span> (7<\/span>, 'Ali'<\/span>, 'Engineering'<\/span>, 4200<\/span>);\nINSERT<\/span> INTO<\/span> emp_salaries VALUES<\/span> (9<\/span>, 'Ibrahim'<\/span>, 'Engineering'<\/span>, 4500<\/span>);\nINSERT<\/span> INTO<\/span> emp_salaries VALUES<\/span> (8<\/span>, 'Mohamed'<\/span>, 'Engineering'<\/span>, 6000<\/span>);\nINSERT<\/span> INTO<\/span> emp_salaries VALUES<\/span> (10<\/span>, 'Ayman'<\/span>, 'Engineering'<\/span>, 5200<\/span>);\nINSERT<\/span> INTO<\/span> emp_salaries VALUES<\/span> (5<\/span>, 'Moemen'<\/span>, 'HR'<\/span>, 3500<\/span>);\nINSERT<\/span> INTO<\/span> emp_salaries VALUES<\/span> (2<\/span>, 'Moetaz'<\/span>, 'HR'<\/span>, 3900<\/span>);\nINSERT<\/span> INTO<\/span> emp_salaries VALUES<\/span> (3<\/span>, 'Abdullah'<\/span>, 'Sales'<\/span>, 4800<\/span>);\nINSERT<\/span> INTO<\/span> emp_salaries VALUES<\/span> (1<\/span>, 'Assem'<\/span>, 'Sales'<\/span>, 5000<\/span>);\nINSERT<\/span> INTO<\/span> emp_salaries VALUES<\/span> (4<\/span>, 'Omar'<\/span>, 'Sales'<\/span>, 4800<\/span>);<\/code><\/span>Code language:<\/span> PostgreSQL SQL dialect and PL\/pgSQL<\/span> (<\/span>pgsql<\/span>)<\/span><\/small><\/pre>\n\n\n
\n
\u26a0\ufe0f If you’re testing in your own environment, you’ll drop a table of the emp_salaries<\/code> name if it does exist. <\/p>\n\n\n\n
Alternatively if you want to skip the table creation task, you can use the CoderPad sandbox at the bottom of this page<\/a> or as a new browser window\/tab<\/a> to run the rest of the tutorial, as the sandbox already has the table loaded for you.<\/p>\n<\/blockquote>\n\n\n\n
PostgreSQL window function syntax<\/strong><\/h2>\n\n\n\n
The syntax of a window function is simple; it looks something like this:<\/p>\n\n\n\n
<function_name>(<argument(s)>) OVER(PARTITION BY <column(s)> ORDER BY <column(s)>) <alias><\/code><\/p>\n\n\n\n
The <function_name><\/code> is an aggregate function like SUM()<\/code>, COUNT()<\/code>, MAX()<\/code>, MIN()<\/code>, AVG()<\/code>, RANK()<\/code>, etc. This function can take a column as an argument and can also take other arguments depending on the use case.<\/p>\n\n\n\n
The OVER()<\/code> clause defines the window size. If it’s empty and PARTITION BY<\/code> and ORDER BY<\/code> are omitted, it means the window is across the whole table. However, PARTITION BY<\/code> partitions the window by the column(s) that follow the clause. In addition, ORDER BY<\/code> sorts the partitioned window by the column(s) indicated by ORDER BY<\/code>.<\/p>\n\n\n\n
PostgreSQL window functions examples<\/strong><\/h2>\n\n\n\n
To get an idea of how to best use window functions, we\u2019ll start with a simple SQL query using an aggregate function:<\/p>\n\n\n
SELECT<\/span> MAX(salary) max_salary\nFROM<\/span> emp_salaries;<\/code><\/span>Code language:<\/span> PostgreSQL SQL dialect and PL\/pgSQL<\/span> (<\/span>pgsql<\/span>)<\/span><\/small><\/pre>\n\n\n
This query outputs the maximum salary across all the records.<\/p>\n\n\n\n
What if you want this max value next to each record in the table?<\/p>\n\n\n\n
You might say this is easy; just add *<\/code> next to the max_salary<\/code> column. However, that query would create the following error:<\/p>\n\n\n
\n
\"\"\/<\/a><\/figure>\n<\/div>\n\n\n
This means that the first column of the emp_salaries<\/code> table — emp_id<\/code> column must appear in the GROUP BY<\/code> clause or be used in an aggregate function.<\/p>\n\n\n\n
To avoid this error, you have two options: <\/p>\n\n\n\n
    \n
  1. A long route using a CTE along with a JOIN<\/code> statement. <\/li>\n\n\n\n
  2. A shorter route using a window function. <\/li>\n<\/ol>\n\n\n\n
    The shorter (and preferred) option for us is to take the window function route and write the following query:<\/p>\n\n\n
    SELECT<\/span> *, MAX(salary) OVER<\/span>() max_salary\nFROM<\/span> emp_salaries;<\/code><\/span>Code language:<\/span> PostgreSQL SQL dialect and PL\/pgSQL<\/span> (<\/span>pgsql<\/span>)<\/span><\/small><\/pre>\n\n\n
    With just OVER()<\/code>, you can use MAX()<\/code> as a window function to aggregate the maximum salary to each record in the table:<\/p>\n\n\n
    \n
    \"\"\/<\/a><\/figure>\n<\/div>\n\n\n
    GROUP BY<\/strong> vs. <\/strong>PARTITION BY<\/strong> clauses<\/strong><\/h3>\n\n\n\n
    What if you want a breakdown of max salaries across each department?<\/p>\n\n\n
    SELECT<\/span> MAX(salary) max_salary\nFROM<\/span> emp_salaries e\nGROUP<\/span> BY<\/span> dep_name;<\/code><\/span>Code language:<\/span> PostgreSQL SQL dialect and PL\/pgSQL<\/span> (<\/span>pgsql<\/span>)<\/span><\/small><\/pre>\n\n\n
    This query gets only the maximum salary across each department:<\/p>\n\n\n
    \n
    \"\"\/<\/a><\/figure>\n<\/div>\n\n\n
    Here, there is one column that represents the max salary per department. Again you wouldn’t easily be able to combine the other columns with the max_salary column unless you use window functions.<\/p>\n\n\n\n
    To compare each record with its corresponding maximum salary by department, use PARTITION BY<\/code>:<\/p>\n\n\n
    SELECT<\/span> *,\n     MAX(salary) OVER<\/span>(PARTITION<\/span> BY<\/span> dep_name) max_salary\nFROM<\/span> emp_salaries;<\/code><\/span>Code language:<\/span> PostgreSQL SQL dialect and PL\/pgSQL<\/span> (<\/span>pgsql<\/span>)<\/span><\/small><\/pre>\n\n\n
    This query results in the following records:<\/p>\n\n\n
    \n
    \"\"\/<\/a><\/figure>\n<\/div>\n\n\n
    As you can see, the aggregate is done over the department names. So you get a breakdown of the max salary of each department next to each record in the emp_salaries<\/code> table.<\/p>\n\n\n\n
    If you\u2019re still confused about the difference between GROUP BY<\/code> and PARTITION BY<\/code>, remember this: GROUP BY<\/code> reduces the number of rows<\/a> returned by rolling them up to calculate the aggregate function for each row. PARTITION BY<\/code> partitions the window based on a specific column; thus it doesn’t affect the number of rows in the result.<\/p>\n\n\n\n
    MAX()<\/strong>, <\/strong>MIN()<\/strong>, <\/strong>AVG()<\/strong>, <\/strong>SUM()<\/strong>, <\/strong>COUNT()<\/strong> functions<\/strong><\/h3>\n\n\n\n
    It’s important to note that the previous examples that you used with MAX()<\/code> can also be applied with either MIN()<\/code>, SUM()<\/code>, or COUNT()<\/code>. These are other aggregate functions and can also be used as window functions as follows:<\/p>\n\n\n
    SELECT<\/span> \n  emp_id, \n  salary, \n  MAX(salary) OVER<\/span>(PARTITION<\/span> BY<\/span> dep_name) max_salary, \n  MIN(salary) OVER<\/span>(PARTITION<\/span> BY<\/span> dep_name) min_salary, \n  SUM(salary) OVER<\/span>(PARTITION<\/span> BY<\/span> dep_name) total_salaries, \n  COUNT(*) OVER<\/span>(PARTITION<\/span> BY<\/span> dep_name) emp_count\nFROM<\/span> emp_salaries;<\/code><\/span>Code language:<\/span> PostgreSQL SQL dialect and PL\/pgSQL<\/span> (<\/span>pgsql<\/span>)<\/span><\/small><\/pre>\n\n\n
    which outputs the following result:<\/p>\n\n\n
    \n
    \"\"\/<\/a><\/figure>\n<\/div>\n\n\n
    \n
    While we don’t cover AVG()<\/code> here, you can find more information on it in the PostgreSQL docs here<\/a>.<\/p>\n<\/blockquote>\n\n\n\n
    ROW_NUMBER()<\/strong> function<\/strong><\/h3>\n\n\n\n
    The ROW_NUMBER()<\/code> is another useful window function. It assigns serial numbers to records according to the window. See the following example:<\/p>\n\n\n
    SELECT<\/span> *, \n  MAX(salary) OVER<\/span>(\n               PARTITION<\/span> BY<\/span> dep_name\n               ) max_salary, \n  ROW_NUMBER() OVER<\/span>() row_num\nFROM<\/span> emp_salaries;<\/code><\/span>Code language:<\/span> PostgreSQL SQL dialect and PL\/pgSQL<\/span> (<\/span>pgsql<\/span>)<\/span><\/small><\/pre>\n\n
    \n
    \"\"\/<\/a><\/figure>\n<\/div>\n\n\n
    As you can see, the row_num column contains an ordered list of numbers across the whole table. That’s because the window here has boundaries across the table, as the OVER()<\/code> clause doesn’t have arguments.<\/p>\n\n\n\n
    A practical use case for ROW_NUMBER()<\/code> is to get the most senior employee in each department. Try it on your own and see what you get.<\/p>\n\n\n
    SELECT<\/span> \n  *, \n  MAX(salary) OVER<\/span>(PARTITION<\/span> BY<\/span> dep_name) max_salary, \n  ROW_NUMBER() OVER<\/span>(PARTITION<\/span> BY<\/span> dep_name) seniority\nFROM<\/span> emp_salaries;<\/code><\/span>Code language:<\/span> PostgreSQL SQL dialect and PL\/pgSQL<\/span> (<\/span>pgsql<\/span>)<\/span><\/small><\/pre>\n\n\n
    Notice that in this case we partition by the department name. That partition would give you a result like this:<\/p>\n\n\n
    \n
    \"\"\/<\/a><\/figure>\n<\/div>\n\n\n
    As you can see, there is an ordered list according to each department. The engineering department has numbers 1<\/code> to 5<\/code>. HR: 1<\/code> and 2<\/code>. Sales: 1<\/code> to 3<\/code>.<\/p>\n\n\n\n
    PARTITION BY<\/strong> and <\/strong>ORDER BY<\/strong> clauses<\/strong><\/h3>\n\n\n\n
    A window function can have ORDER BY<\/code> inside an OVER()<\/code> clause. To see what that looks like, assume that emp_id<\/code> represents seniority and that an emp_id<\/code> of 8<\/code> precedes 9<\/code>, meaning that the former joined the company before the latter. We will then sort according to the emp_id<\/code> to see the first employees who joined each department:<\/p>\n\n\n
    SELECT<\/span> *, \n  MAX(salary) OVER<\/span>(\n               PARTITION<\/span> BY<\/span> dep_name\n               ) max_salary, \n  ROW_NUMBER() OVER<\/span>(\n    PARTITION<\/span> BY<\/span> dep_name \n    ORDER<\/span> BY<\/span> emp_id\n    ) seniority \nFROM<\/span> emp_salaries;<\/code><\/span>Code language:<\/span> PostgreSQL SQL dialect and PL\/pgSQL<\/span> (<\/span>pgsql<\/span>)<\/span><\/small><\/pre>\n\n\n
    The seniority column represents ordered numbers according to the emp_id<\/code>. If you filter by seniority with 1<\/code> values, you get the first employee for each department who joined the company.<\/p>\n\n\n\n
    Give it a try here using a subquery:<\/p>\n\n\n
    SELECT<\/span> * \nFROM<\/span> (\n    SELECT<\/span> *, \n         MAX(salary) OVER<\/span>(\n                      PARTITION<\/span> BY<\/span> dep_name\n                      ) max_salary, \n         ROW_NUMBER() OVER<\/span>(\n                      PARTITION<\/span> BY<\/span> dep_name \n                      ORDER<\/span> BY<\/span> emp_id\n                      ) seniority \n    FROM<\/span> emp_salaries\n  ) e \nWHERE<\/span> e.seniority < 2<\/span>;<\/code><\/span>Code language:<\/span> PostgreSQL SQL dialect and PL\/pgSQL<\/span> (<\/span>pgsql<\/span>)<\/span><\/small><\/pre>\n\n\n
    RANK()<\/strong> function<\/strong><\/h3>\n\n\n\n
    The RANK()<\/code> window function returns the rank of the current row with gaps. But what do we mean by gaps?<\/p>\n\n\n
    SELECT<\/span> *, \n  RANK() OVER<\/span>(\n          PARTITION<\/span> BY<\/span> dep_name \n          ORDER<\/span> BY<\/span> salary DESC<\/span>\n          ) rank \nFROM<\/span> emp_salaries;<\/code><\/span>Code language:<\/span> PostgreSQL SQL dialect and PL\/pgSQL<\/span> (<\/span>pgsql<\/span>)<\/span><\/small><\/pre>\n\n\n
    This query returns the ranks of every employee, in descending order, according to each department. Gaps exist when there are duplicate values:<\/p>\n\n\n
    \n
    \"\"\/<\/a><\/figure>\n<\/div>\n\n\n
    As you can see, Ayman<\/code> and Ahmed<\/code> share the same rank (2<\/code>) on engineering department salaries. Rank 3<\/code> is a gap, and the RANK()<\/code> function skips it. The next rank is 4<\/code> taken by Ibrahim<\/code>.<\/p>\n\n\n\n
    DENSE_RANK()<\/strong> function<\/strong><\/h3>\n\n\n\n
    DENSE_RANK()<\/code> window function is similar to RANK()<\/code> except that it returns the rank without gaps. If you replace RANK()<\/code> with DENSE_RANK()<\/code> in the previous query, you’ll find a result query like the following:<\/p>\n\n\n
    \n
    \"\"\/<\/a><\/figure>\n<\/div>\n\n\n
    Now, rank 3<\/code> is not skipped, as in the case of RANK()<\/code>.<\/p>\n\n\n\n
    Try it out!<\/strong><\/h2>\n\n\n\n\t