Introduction to NumPy

Written and reviewed by Rohit Singh, Computer Science & Commerce faculty at UniDrill (MCA, Osmania University · 6 years teaching). Based on NCERT Informatics Practices Class XI, Chapter 6.

Why can't a plain list do this?

Suppose you have the marks of 40 students and you want to add 5 grace marks to everyone. With a Python list you must write a loop. With a NumPy array you write marks + 5 and every element changes at once. That one idea, do the same thing to every element without a loop, is why NumPy sits underneath almost all data work in Python, including Pandas in Class XII.

NumPy stands for Numerical Python. It is a library for data analysis and scientific computing, and its core object is the ndarray (n-dimensional array), usually just called an array. You install it with pip install numpy and load it with import numpy as np. The alias np is a convention, not a rule, but the package name itself must be written in lowercase.

Array versus list

An array stores many values of the same data type under one name, packed side by side in memory, and each value is reached by a zero-based index. A list is far more relaxed, and that freedom is exactly what makes it slower for number crunching.

Why contiguous memory makes it fast. Because all elements have the same size and sit next to each other, the computer can jump straight to any element and sweep through the block without chasing scattered locations. Think of a train of identical coaches coupled together versus coaches parked at random places across a city.

Beyond the textbook. For the exam: an array holds one data type and is stored contiguously, so it is faster than a list. In reality: the speed also comes from NumPy running its loops in compiled C code instead of Python. And an array can be sliced without copying the data (a slice is a window onto the same memory, called a view), so changing a slice changes the original array. A list slice, by contrast, is a copy. Why it matters: if a program mysteriously changes data you did not touch, a shared view is the usual culprit.

Creating arrays from lists

The array() function converts a list into an array. A flat list gives a 1-D array (one row of elements). A nested list, meaning a list whose elements are lists, gives a 2-D array.

Notice that the integers in array3 were turned into floats (3 became 3.0). An array has only one type, so NumPy promotes every element to the "widest" type present. With a string in the list, everything becomes a string.

A classic slip is np.array(1, 2, 3, 4). That passes four arguments. The function needs one argument that is a list, so the correct form is np.array([1, 2, 3, 4]).

Attributes: what is inside this array?

Five attributes describe an array. They are properties, so they are written without brackets: a.shape, not a.shape().

A 2-D array has two axes. Axis 0 runs down the rows and axis 1 runs across the columns. A 1-D array's shape has a trailing comma, (3,), because a Python tuple with one item needs it.

Worked example: reading the attributes. For a = np.array([[1.5, 2, 3], [4, 5, 6]]), find ndim, shape, size, dtype and itemsize.

The common wrong step is to say itemsize is 64. That is the number of bits; itemsize is always in bytes. Likewise int32 and float32 give 4 bytes and int64 and float64 give 8 bytes.

Beyond the textbook. For the exam: an integer array shows int32 and itemsize 4. In reality: the default integer type depends on your machine and NumPy version, so the same code can print int64 and itemsize 8 on another computer. Write dtype=int32 or dtype=int64 explicitly if the exact size matters. Why it matters: never hard code an itemsize in your head. Work it out from the dtype named in the question.

Other ways to create arrays

For zeros() and ones() the shape goes in as one tuple, so there are double brackets: np.zeros((3, 4)). Pass dtype=int to get integer zeros or ones. In arange() the stop value is never included, exactly like range(), so np.arange(2, 10, 3) gives [2 5 8]: the next value, 11, is past the stop.

Pause and predict: what does np.arange(1, 10, 3) give, and what is its size?

Check your answer: Start at 1, step 3, stop before 10: [1 4 7]. The next value would be 10, which is the excluded stop. So the size is 3.

Indexing and slicing

Indexing picks single elements. For a 2-D array you give two indices, array[row, column], and both start at 0. Going past the end raises an IndexError.

Slicing picks a block using start:end, where the end index is excluded and a missing start or end means "from the beginning" or "to the end". ::-1 reverses. For a 2-D array you slice each axis, separated by a comma: array[row_slice, column_slice]. A bare : means "everything along this axis".

Worked example: slicing a marks table. The marks of four students in Maths, English and Science are stored as shown.

The wrong step to avoid: reading marks[1:3, 0:2] as "rows 1 to 3, columns 0 to 2", which would give three rows and three columns. The end is excluded on both axes, so the result has $3 - 1 = 2$ rows and $2 - 0 = 2$ columns. A quick check: the shape of a slice is (end minus start) along each axis.

Pause and predict: for the same marks array, what is marks[0, 4]?

Check your answer: An IndexError. There are only 3 columns, so the valid column indices are 0, 1 and 2. Index 4 is out of bounds for axis 1.

Arithmetic on arrays

When you use +, -, *, /, % or ** on two arrays of the same shape, NumPy works on each corresponding pair of elements. This is called an element wise operation. With a single number on one side (marks + 5, marks / 3) the number is applied to every element.

Worked example: element wise versus matrix product. Let array1 = np.array([[3, 6], [4, 2]]) and array2 = np.array([[10, 20], [15, 12]]).

The common wrong step is to treat * as matrix multiplication. In NumPy, * is always element wise. The matrix product uses the @ operator. Check the first cell both ways: element wise gives $3 \times 10 = 30$, while the matrix product gives $3 \times 10 + 6 \times 15 = 120$.

If the shapes differ, for example a 2 by 2 array with a 3 by 3 array, the operation fails with an error. The textbook rule to remember is that the shapes must be equal.

Transpose, sort, concatenate, reshape and split

Transpose swaps rows and columns, so a shape (3, 4) becomes (4, 3). Write a.transpose() or a.T. The original array is not changed.

Sort arranges values in ascending order. For a 2-D array the default is row wise (axis=1, each row sorted on its own), and axis=0 sorts each column on its own. Here sort() changes the array itself and returns nothing.

Re-creating x and calling x.sort(axis=0) instead gives [[-5 -7 -1 4] [10 1 0 20] [30 1 200 40]], where every column is now ascending from top to bottom.

Concatenate joins arrays with np.concatenate((a, b), axis). The default is axis=0 (stack below). All dimensions must match except along the axis you join. Reshape changes the shape with a.reshape(r, c) but cannot change how many elements there are. Split cuts an array into pieces with np.split(a, indices_or_N, axis): give a list of cut positions, or an integer N for N equal parts (the default axis is 0).

Worked example: when does concatenation fail? a has shape (2, 2) and b = np.zeros((2, 3), dtype=a.dtype) has shape (2, 3).

The memory trick: the axis you join along is the one that is allowed to differ; every other axis must be equal.

Worked example: reshape and split. Take arr = np.arange(10, 22), which has 12 elements.

For a 2-D array, np.split(a, [1, 3]) cuts rows at indices 1 and 3, giving three pieces. Adding axis=1 cuts at column indices instead. The pieces come back as a list, so you can unpack them: first, second, third = np.split(a, [1, 3]).

Beyond the textbook. For the exam: arrays cannot be expanded, so concatenate is how you "add" elements. In reality: concatenate builds a brand new array and copies the data into it, which is why growing an array one element at a time inside a loop is slow. Collect values in a list first, then convert once. Also, a.sort() changes a in place and returns None, while the function np.sort(a) returns a sorted copy and leaves a as it was. Why it matters: b = a.sort() leaves b empty, a mistake that shows up as a famous MCQ trap.

Statistics and the axis rule

NumPy gives max(), min(), sum(), mean() and std(). With no argument they work on the whole array and return one number. With axis they work along that axis.

The rule that never fails: the axis you name is the one that disappears.

Worked example: statistics on the marks array. Using the 4 by 3 marks array from earlier:

The wrong step is to say "axis=0 means row wise". It does not; axis=0 gives column results because it collapses the row axis. A shape check helps: for a (4, 3) array, axis=0 returns 3 numbers and axis=1 returns 4.

Beyond the textbook. For the exam: std() is the standard deviation of the values. In reality: NumPy's std() divides by $N$ (the number of values), the population formula. Many calculators and the Pandas std() method divide by $N - 1$ instead, so you can get a slightly different number for the same data. Why it matters: when two tools disagree on a standard deviation, this is usually the reason.

Pause and predict: for B = np.array([[3, 6], [4, 2]]), what do B.sum(axis=0) and B.mean(axis=1) give?

Check your answer: axis=0 adds down the columns: $3+4 = 7$ and $6+2 = 8$, so [7 8]. axis=1 averages across the rows: $(3+6)/2 = 4.5$ and $(4+2)/2 = 3$, so [4.5 3. ].

Loading and saving data files

Real data lives in files, most often CSV (comma separated values) text files. Every row must have the same number of values.

Two defaults catch students out. The default delimiter is whitespace (so for a comma file you must say delimiter=','), and the default dtype is float, so you get 36. unless you ask for dtype=int.

unpack=True transposes the loaded data so each column becomes its own array. For a file with columns RollNo, Marks1, Marks2 you can write rollno, mks1, mks2 = np.loadtxt('data.txt', skiprows=1, delimiter=',', unpack=True, dtype=int). Without it you get rows instead.

Missing values. loadtxt() stops with an error when a field cannot be converted. genfromtxt() carries on and fills the gap.

So a row 2,ab,23,45 loaded as floats becomes 2. nan 23. 45., and a row 3,43,51, ends with nan.

Pause and predict: a file has the header line RollNo,Marks, and you call np.loadtxt('f.txt', delimiter=',') with no skiprows. What happens?

Check your answer: It fails, because the header contains text that cannot be converted to a number. Adding skiprows=1 tells NumPy to ignore that first line.

Function sheet

Exam traps

60-second revision

Where this chapter connects

NumPy arrays are the engine behind the Series and DataFrame objects in Data Handling Using Pandas - I, where the same ideas of shape, axis and element wise arithmetic return with row and column labels added. The descriptive statistics here (mean, standard deviation) come from Understanding Data. For working with Python's own list structure, see Lists, and for turning arrays into charts, see Plotting Data using Matplotlib.