pandas DataFrame

A DataFrame is a table: rows and columns, like a sheet in Excel, where every column is a pandas Series and all columns share the same row index. It is the object you will use in almost every lesson from now on.

In this lesson you will create DataFrames, look at their parts, select and add columns, sort, rename and drop. In the next lesson you will load DataFrames from files instead of typing them.

Creating a DataFrame

The most common way to build a small DataFrame by hand is a dictionary of lists: each key becomes a column name and each list becomes that column's values.

import pandas as pd

students = pd.DataFrame({
    "name": ["Aarav", "Priya", "Kabir", "Meera", "Rohan"],
    "city": ["Rohtak", "Jaipur", "Rohtak", "Kochi", "Pune"],
    "maths": [78, 91, 55, 84, 69],
    "science": [82, 88, 61, 79, 74],
})
print(students)
print(students.shape)
print(students.columns.tolist())
print(students.dtypes)

    name    city  maths  science
0  Aarav  Rohtak     78       82
1  Priya  Jaipur     91       88
2  Kabir  Rohtak     55       61
3  Meera   Kochi     84       79
4  Rohan    Pune     69       74
(5, 4)
['name', 'city', 'maths', 'science']
name         str
city         str
maths      int64
science    int64
dtype: object

The numbers on the left (0 to 4) are the index, created automatically. shape gives (rows, columns), columns holds the column names and dtypes shows each column's type. Each column has one dtype, but different columns can have different types: that is what makes a DataFrame a good fit for real-world tables.

You can also build a DataFrame from a list of dictionaries (one per row), which is how data often arrives from a web API, or from a 2-D NumPy array plus column names.

import numpy as np
import pandas as pd

orders = [
    {"order_id": "A101", "item": "Kurta", "qty": 2, "price": 799},
    {"order_id": "A102", "item": "Tawa", "qty": 1, "price": 899},
    {"order_id": "A103", "item": "Earbuds", "qty": 1},
]
print(pd.DataFrame(orders))

readings = np.array([[31.2, 24.0], [33.5, 25.1], [30.8, 23.6]])
print(pd.DataFrame(readings, columns=["max_temp", "min_temp"], index=["Mon", "Tue", "Wed"]))

  order_id     item  qty  price
0     A101    Kurta    2  799.0
1     A102     Tawa    1  899.0
2     A103  Earbuds    1    NaN
     max_temp  min_temp
Mon      31.2      24.0
Tue      33.5      25.1
Wed      30.8      23.6

The third order has no price key, so pandas fills that cell with NaN. With the NumPy array, you supplied both column names and an index of day names.

Selecting columns

Square brackets with one column name return a Series. Square brackets with a list of names return a smaller DataFrame. The double brackets are a list inside the selector, not a special syntax.

import pandas as pd

students = pd.DataFrame({
    "name": ["Aarav", "Priya", "Kabir", "Meera", "Rohan"],
    "city": ["Rohtak", "Jaipur", "Rohtak", "Kochi", "Pune"],
    "maths": [78, 91, 55, 84, 69],
    "science": [82, 88, 61, 79, 74],
})

maths = students["maths"]
print(type(maths).__name__, maths.mean())

subset = students[["name", "maths"]]
print(type(subset).__name__)
print(subset)

Series 75.4
DataFrame
    name  maths
0  Aarav     78
1  Priya     91
2  Kabir     55
3  Meera     84
4  Rohan     69

You may also see students.maths (attribute access). It works for simple names, but fails for names with spaces or names that clash with methods (a column called count or size), so square brackets are the safer habit.

Adding and changing columns

Assigning to a new column name creates it. Calculations are vectorised, so the new column is computed for every row at once.

import pandas as pd

students = pd.DataFrame({
    "name": ["Aarav", "Priya", "Kabir", "Meera", "Rohan"],
    "maths": [78, 91, 55, 84, 69],
    "science": [82, 88, 61, 79, 74],
})

students["total"] = students["maths"] + students["science"]
students["percent"] = students["total"] / 200 * 100
students["result"] = "Pass"
print(students)

ranked = students.assign(rank=students["total"].rank(ascending=False).astype(int))
print(ranked.sort_values("rank")[["name", "total", "rank"]])

    name  maths  science  total  percent result
0  Aarav     78       82    160     80.0   Pass
1  Priya     91       88    179     89.5   Pass
2  Kabir     55       61    116     58.0   Pass
3  Meera     84       79    163     81.5   Pass
4  Rohan     69       74    143     71.5   Pass
    name  total  rank
1  Priya    179     1
3  Meera    163     2
0  Aarav    160     3
4  Rohan    143     4
2  Kabir    116     5

Assigning a single value, like "Pass", fills the whole column. assign() is the method version: it returns a new DataFrame with the extra column and leaves the original unchanged, which makes it good for chaining several steps together.

Sorting, renaming, dropping and the index

These four tidy-up operations appear in nearly every analysis. Each returns a new DataFrame.

import pandas as pd

shops = pd.DataFrame({
    "Shop Name": ["Sharma Sweets", "Kochi Spice Co", "Pune Books", "Jaipur Gems"],
    "City": ["Rohtak", "Kochi", "Pune", "Jaipur"],
    "Sales_2025": [18.5, 22.1, 9.8, 31.4],
    "notes": ["", "new branch", "", ""],
})

clean = (
    shops
    .rename(columns={"Shop Name": "shop", "City": "city", "Sales_2025": "sales_lakh"})
    .drop(columns="notes")
    .sort_values("sales_lakh", ascending=False)
)
print(clean)

by_shop = clean.set_index("shop")
print(by_shop.loc["Pune Books", "sales_lakh"])
print(by_shop.reset_index().head(2))

             shop    city  sales_lakh
3     Jaipur Gems  Jaipur        31.4
1  Kochi Spice Co   Kochi        22.1
0   Sharma Sweets  Rohtak        18.5
2      Pune Books    Pune         9.8
9.8
             shop    city  sales_lakh
0     Jaipur Gems  Jaipur        31.4
1  Kochi Spice Co   Kochi        22.1

Wrapping the steps in brackets lets you write one operation per line, a style called method chaining. After sort_values(), the original index numbers stay with their rows (3, 1, 0, 2), which shows they are labels, not positions. set_index("shop") turns a column into the index so you can look rows up by name with .loc, and reset_index() turns it back into a normal column.

From DataFrame back to NumPy

When you need raw numbers, for example for a NumPy calculation or a machine learning library, use to_numpy().

import pandas as pd

df = pd.DataFrame({"maths": [78, 91, 55], "science": [82, 88, 61]})
arr = df.to_numpy()
print(arr)
print(arr.shape, arr.mean(axis=0))

[[78 82]
 [91 88]
 [55 61]]
(3, 2) [74.66666667 77.        ]

You lose the labels, but you get a plain 2-D array to which everything from the NumPy lessons applies.

Common mistakes

• Single brackets for several columns. df["name", "maths"] raises a KeyError. Use a list: df[["name", "maths"]].
• Forgetting to assign the result. df.sort_values("total") returns a sorted copy; df itself is unchanged unless you write df = df.sort_values("total").
• Treating the index as row positions after sorting. Index labels move with their rows. Use .iloc for positions or reset_index(drop=True) to renumber.
• Attribute access for every column. df.count is a method, not your count column. Prefer df["count"].

Key points

• A DataFrame is a table of columns (each a Series) sharing one index.
• Create one from a dict of lists, a list of dicts or a 2-D NumPy array with column names.
• df["col"] returns a Series; df[["a", "b"]] returns a DataFrame.
• New columns come from vectorised calculations; assign() returns a new DataFrame for chaining.
• rename(), drop(), sort_values(), set_index() and reset_index() return new DataFrames.

import pandas as pd

shop = pd.DataFrame({
    "product": ["Masala Chai", "Basmati Rice", "Ghee", "Almonds"],
    "price": [180, 649, 690, 520],
    "units_sold": [120, 45, 38, 52],
})
shop["revenue"] = shop["price"] * shop["units_sold"]
top = shop.sort_values("revenue", ascending=False)
print(top[["product", "revenue"]])
print(top.set_index("product").loc["Ghee", "revenue"])

        product  revenue
1  Basmati Rice    29205
3       Almonds    27040
2          Ghee    26220
0   Masala Chai    21600
26220