pandas for Analysts

By the end of this lesson you will be able to load a CSV file into pandas, inspect it, pick columns, filter rows, sort, add a calculated column and count values. For each step you will see the SQL you already know, so pandas feels familiar rather than new.

pandas is a Python library for working with tables. A table in pandas is called a DataFrame: rows and columns, like a spreadsheet sheet or a SQL table, but controlled by code. Why use it when you have Excel and SQL? Because a pandas script can clean a messy file in seconds, handle files too big for Excel, and be rerun next month with one click. Think of it as a recipe card: once written, anyone can cook the same dish again.

The full Data Science course teaches pandas in depth. This lesson covers the core an analyst uses daily.

Step 1: load and look

pd.read_csv reads a CSV file into a DataFrame. Then look at it three ways: its size, its first rows and its column types.

import pandas as pd

sales = pd.read_csv("sales.csv")

print(sales.shape)          # (rows, columns)
print(sales.head(3))        # first three rows
print(sales.dtypes)         # type of each column

(1217, 10)
   order_id  order_date     city  ... unit_price discount payment_mode
0  ORD10001  2025-01-01   Mumbai  ...        699        0          UPI
1  ORD10002  2025-01-01  Kolkata  ...       1599        5         Card
2  ORD10003  2025-01-01   Mumbai  ...        699        0          NaN

[3 rows x 10 columns]
order_id            str
order_date          str
city                str
state               str
category            str
product             str
quantity        float64
unit_price        int64
discount          int64
payment_mode        str
dtype: object

What you should see: 1,217 rows and 10 columns, three sample orders, and a type for each column. str means text, int64 whole numbers and float64 decimal numbers. Notice that quantity is float64 even though quantities are whole numbers. That happens because the column has blanks, and pandas stores a blank number as NaN (Not a Number), which needs a decimal type. Small clues like this tell you about the data.

Step 2: select columns and filter rows

Pick columns with a list inside square brackets. Filter rows with a boolean mask: a True/False test for every row. Business question: which UPI orders from Kochi had a discount?

import pandas as pd

sales = pd.read_csv("sales.csv")

mask = (sales["city"] == "Kochi") & (sales["payment_mode"] == "UPI") & (sales["discount"] > 0)
kochi_upi = sales.loc[mask, ["order_id", "order_date", "product", "unit_price", "discount"]]

print(len(kochi_upi), "orders")
print(kochi_upi.head())

10 orders
     order_id  order_date           product  unit_price  discount
162  ORD10162  2025-02-27  Basmati Rice 5kg         649         5
403  ORD10402  2025-05-24        Silk Saree        1899         5
603  ORD10599  2025-08-02        Story Book         199        15
623  ORD10619  2025-08-07     Running Shoes        2499        10
649  ORD10645  2025-08-15  Basmati Rice 5kg         649        20

What you should see: a count and the first five matching orders. Combine conditions with & (and) and | (or), and put each condition in round brackets. Python's words and and or do not work here, which is a very common beginner error.

Here is how pandas lines up with SQL:

Step 3: add calculated columns

New columns are calculated for every row at once; no loop is needed. The discount column holds a percentage, so the money received is quantity times price times (1 minus discount/100).

import pandas as pd

sales = pd.read_csv("sales.csv")
sales["gross"] = sales["quantity"] * sales["unit_price"]
sales["revenue"] = sales["gross"] * (1 - sales["discount"] / 100)
sales["order_date"] = pd.to_datetime(sales["order_date"])
sales["weekday"] = sales["order_date"].dt.day_name()

print(sales[["order_id", "quantity", "unit_price", "discount", "gross", "revenue", "weekday"]].head())

   order_id  quantity  unit_price  discount   gross  revenue    weekday
0  ORD10001       1.0         699         0   699.0   699.00  Wednesday
1  ORD10002       1.0        1599         5  1599.0  1519.05  Wednesday
2  ORD10003       1.0         699         0   699.0   699.00  Wednesday
3  ORD10004       1.0        1499         0  1499.0  1499.00   Thursday
4  ORD10005       1.0         399         0   399.0   399.00   Thursday

What you should see: five rows with the new columns. pd.to_datetime turns the text dates into real dates, which unlocks helpers under .dt such as day_name(), month and to_period("M"). Always convert dates before working with them.

Step 4: sort and count

Business question: what are the five most expensive single orders, and which payment modes do people use?

import pandas as pd

sales = pd.read_csv("sales.csv")
sales["revenue"] = sales["quantity"] * sales["unit_price"] * (1 - sales["discount"] / 100)

top5 = sales.sort_values("revenue", ascending=False).head(5)
print(top5[["order_id", "city", "product", "quantity", "revenue"]])
print()
print(sales["payment_mode"].value_counts(dropna=False))

      order_id       city           product  quantity  revenue
1020  ORD11014       Pune        Exam Guide     120.0  51300.0
719   ORD10715     Mumbai        Smartwatch       4.0  11396.2
99    ORD10100      Delhi  Masala Chai 250g      60.0  10800.0
1068  ORD11062     Jaipur        Smartwatch       4.0  10796.4
560   ORD10556  Hyderabad     Running Shoes       4.0   9996.0

payment_mode
UPI                 608
Card                207
Cash on Delivery    183
Wallet              100
Net Banking          90
NaN                  11
UPI                   4
COD                   4
upi                   3
card                  2
Upi                   2
wallet                2
CARD                  1
Name: count, dtype: int64

What you should see: the top five orders, led by two very large quantities, and a payment count with messy names like upi, COD and blanks (NaN). dropna=False makes value_counts show the blanks too, which you almost always want while exploring. Those messy names will be fixed in A Data Cleaning Checklist.

Step 5: summarise numbers quickly

describe() gives count, mean, spread and percentiles for every number column in one line. It is the fastest way to spot impossible values.

import pandas as pd

sales = pd.read_csv("sales.csv")
print(sales[["quantity", "unit_price", "discount"]].describe().round(1))

       quantity  unit_price  discount
count    1211.0      1217.0    1217.0
mean        1.6      1122.8       5.0
std         3.9       815.0       6.5
min        -1.0       180.0       0.0
25%         1.0       520.0       0.0
50%         1.0       899.0       0.0
75%         2.0      1499.0      10.0
max       120.0      3299.0      20.0

What you should see: the minimum quantity is negative and the maximum is far above the 75th percentile. A quantity of -1 is impossible for a sale, and a quantity in the hundreds is unusual for an online shop. describe() found both problems in one line.

Common mistakes

• Using and/or in masks. Use & and |, with each condition in brackets.
• Forgetting to convert dates. Text dates sort correctly only in YYYY-MM-DD form and have no .dt helpers.
• Chained assignment. df[mask]["col"] = value never changes df in pandas 3.0, only shows a warning; use df.loc[mask, "col"] = value.
• Hiding blanks. Use value_counts(dropna=False) and isna().sum() while exploring.

Key points

• A DataFrame is a table; read_csv, shape, head and dtypes are the first four things to run.
• Filter with boolean masks using &, | and .isin(); select columns with a list.
• New columns are calculated for all rows at once; convert dates with pd.to_datetime.
• sort_values, value_counts and describe answer many quick questions.
• pandas 3.0 uses a str type for text and Copy-on-Write; use .loc to change values.

import pandas as pd

emp = pd.read_csv("employees.csv")
mask = (emp["department"] == "Sales") & emp["work_mode"].isin(["Remote", "Hybrid"]) & (emp["rating"] >= 4)
result = emp.loc[mask, ["name", "city", "work_mode", "rating"]].sort_values(["rating", "name"], ascending=[False, True])
print(len(result))
print(result)