File Handling in Python

Written and reviewed by Rohit Singh, Computer Science & Commerce faculty at UniDrill (MCA, Osmania University · 6 years teaching). Based on NCERT Computer Science Class XII, Chapter 2.

Where does your data go when the program ends?

Imagine a marks entry program. You type in 40 students' marks, close the program, and open it the next day. Everything is gone. Variables live in memory only while the program runs. To keep data for later, a program must store it in a file: a named location on secondary storage (hard disk, pen drive, SSD) where data stays permanently.

This chapter follows the life of a file in Python: open it, read or write it, move around in it, and close it. Then it adds one powerful trick, the pickle module, which stores whole Python objects such as lists and records.

Text files and binary files

Every file stores bytes, but what the bytes mean differs.

Why binary files break so easily. A binary file has a precise structure that its own software expects. A JPEG image, for instance, is a carefully ordered stream of bytes. Change one bit in the wrong place and the software can no longer decode it. Text has no hidden structure to break, which is why you can read a damaged text file in any editor.

Beyond the textbook. For the exam: in a text file each line ends with the single EOL character \n. In reality: Windows stores line endings as two characters, but Python quietly converts them to \n when it reads a text file and converts back when it writes. Your code sees the same \n on every system.

Opening a file

The built in function open() links your program to a file:

file_object = open(file_name, access_mode)

It returns a file handle, stored in file_object, which you use for every later operation. The second argument is optional: if you leave it out, the file opens in read mode (r) as a text file. The file object carries three useful attributes.

The access mode decides what you may do and where the offset (the current position) starts. This table is the most tested one in the chapter.

Two quick rules cover the whole table. Modes that start with r never create a file, so they need it to exist. Modes with w always erase, so a careless open(name, "w") on an existing file destroys its contents. Binary versions simply add b, and wb and ab are the modes used with pickle.

Worked example: what is in the file? The file a.txt is opened and closed three times in a row. What does it contain at the end?

The wrong step is to read the third w as "write more". Write mode starts from nothing every time.

Closing a file, and the with clause

file_object.close() pushes any unsaved data to the file and frees the memory used. This matters because write() first puts data in a buffer (a holding area in memory). The data reaches permanent storage only when the file is closed, or when you call flush() to force it out. A program that crashes before closing can lose data still sitting in the buffer.

The with clause removes the risk:

The file is closed automatically when control leaves the with block, even if an exception occurs inside it. No explicit close() is needed, which is why modern code prefers with. It does the job of a try with a finally that closes the file.

Writing to a text file

Both methods need strings. To write a number, convert it first with str(), otherwise Python raises TypeError. Neither method adds \n for you, so you put line breaks in yourself.

Worked example: write and writelines. What does this program print, and what ends up in the file?

write("Hi\n") returns 3, counting H, i and the newline. The next two calls add no line breaks, so everything joins up.

Reading from a text file

Suppose poem.txt holds three lines, with no newline after the last one:

Notice that readlines() keeps the \n at the end of each line, while splitlines() removes them. The last line has no \n only because the file does not end with one. When the end of the file is reached, readline() returns an empty string '', not None. That empty string is how a loop knows to stop.

The loop is true while line holds text. An empty string counts as false, so the loop ends at the end of the file.

Pause and predict: after two readline() calls on poem.txt, what does read(4) return?

Check your answer: 'This'. Each readline() moves the offset past its line, so after two calls the offset is at the start of line 3, and read(4) takes the first four characters of it.

tell() and seek(): moving inside a file

A file object remembers its offset, the number of bytes from the beginning where the next read or write will happen. tell() reports it. seek(offset, reference_point) moves it. The reference point is optional and defaults to 0.

Worked example: tracing the offset. Using poem.txt from above:

Everything is counted in bytes, never in lines. tell() after reading line 1 is 15, not 1. This is why reading a file twice needs seek(0) in between: after the first full read the offset sits at the end.

Beyond the textbook. For the exam: seek(offset, reference_point) accepts 0, 1 and 2 as reference points. In reality: a file opened in text mode only allows seek(offset, 0) freely. Moving by a non zero offset from the current position or from the end raises an error, and only binary files accept all three reference points. For the same reason, the offsets you pass to seek() in text mode should come from earlier tell() calls or be 0.

Writing and then reading the same file

Mode w+ creates (or erases) a file and lets you both write and read. After writing, the offset is at the end, so seek(0) is needed before reading back.

A short program can then total the marks. Each line is split into words with split(), and the second word is converted to a number.

Pause and predict: a file data.txt contains Hello. It is opened with open("data.txt", "a+"), and the very next statement is print(f.read()). What is printed?

Check your answer: An empty line. Mode a+ starts the offset at the end of the file, so read() finds nothing to read. A seek(0) first would let it read Hello.

The pickle module: saving whole objects

A text file can only hold text, so a record such as [101, "Asha", 95] would come back as a string, not as a list. Python's pickle module solves this. Pickling (serialisation) turns a Python object into a stream of bytes that can be stored in a binary file or sent over a network. Unpickling (de-serialisation) turns the bytes back into the original object, with the same structure and data types.

The steps never change. Import pickle, open the file in binary mode, call dump() or load(), then close it.

Each dump() writes one object, so several records need a loop. Each load() reads the next one in the same order. When the objects run out, load() raises EOFError. This is exactly why the reading loop sits inside a try with except EOFError: the error is the signal for "no more records", which links this chapter directly to Exception Handling.

Worked example: how many records? The program above runs three times. How many records does emp.dat hold after each run, if the open mode changes as shown?

Beyond the textbook. For the exam: pickle stores any Python object in a binary file. In reality: never unpickle a file that came from someone you do not trust. Loading a pickle can run code hidden inside it, so pickle is meant only for data your own programs wrote. For data you exchange with other people or systems, formats such as CSV are the safer choice.

Exam traps

60-second revision

Where this chapter connects

The finally clause and the with statement are two ways to guarantee a file closes, so revisit Exception Handling in Python. Later chapters read data back from files too, for example in Data Handling using Pandas I.