You've got data sitting in a text file and need to process it with Python. Maybe it's a log file, a CSV export, or configuration settings. Whatever the source, Python makes working with text files straightforward once you understand a few core concepts.
The key is learning how paths work, which access mode to choose, and which reading method fits your goal. Codeunderscored covers these fundamentals in detail, and we'll walk through everything you need to start reading files confidently.
What happened
- Text files store data as readable characters with line terminators
- File paths point to file locations using directory structure
- Python's open() function handles file access with different modes
- Reading methods include read(), readline(), and readlines()
- The with statement ensures files close properly
Understanding Files and Paths
A file path is a string that tells Python exactly where a file lives on your system. It breaks down into three parts: the folder location, the filename, and the extension that identifies the file type.
Paths include directory names separated by slashes. On Windows you might see backslashes, while Mac and Linux use forward slashes. Python usually handles both formats, so you don't have to worry too much about cross-platform differences.
If your file sits in the same folder as your Python script, you only need the filename. Otherwise, specify the full path from your root directory.
Text Files vs Binary Files
Python works with two kinds of files. Text files store information as readable characters, with each line ending in a special newline character ('\n'). Binary files store raw bytes without any line structure.
Most everyday files you encounter are text files: documents, code files, CSV exports, and log files. Binary files include images, executables, and compressed archives.
This guide focuses on text files, which is what most programming tasks involve.
Choosing the Right Access Mode
When you open a file, you choose how Python will interact with it. The access mode controls what operations are allowed.
Read mode ('r') opens a file for reading only. The file handle starts at the beginning. If the file doesn't exist, Python raises an error.
Write mode ('w') opens a file for writing. It erases existing content. If the file doesn't exist, Python creates it.
Append mode ('a') opens a file for writing. New content goes after existing data. The file handle starts at the end.
Read-write mode ('r+') allows both reading and writing. The handle starts at the beginning. Write operations don't erase existing content, but they overwrite characters at the current position.
Most beginners stick with 'r' for reading and 'w' for writing.
Reading Methods Compared
Python gives you three ways to read text files, each suited for different situations.
The read() method loads the entire file into one string:
“`python
With open('data.txt', 'r') as f:
Content = f.read()
“`
This approach works well when you need to process the whole file at once, search for patterns, or manipulate the content as a single text block.
The readlines() method returns a list where each element is one line from the file:
“`python
With open('data.txt', 'r') as f:
Lines = f.readlines()
“`
This gives you line-by-line access while keeping everything in memory. Useful when you need random access to specific lines.
Iteration is the most memory-efficient approach for large files:
“`python
With open('data.txt', 'r') as f:
For line in f:
Print(line.strip())
“`
This processes one line at a time, never loading the entire file into RAM. For log files or large datasets, this matters.
The with statement handles file closure automatically, even if your code hits an error midway. Always use it.
Our Analysis
Python's file handling is refreshingly simple compared to lower-level languages. The three reading methods cover most needs: load everything for small files, iterate for large ones.
The access modes matter more than they first appear. Choosing 'w' by accident on an existing file destroys data silently. 'r' is safer because it refuses to create files, forcing you to be intentional.
Memory management through iteration is the real win. Reading gigabyte log files line-by-line works on modest hardware. Loading the same file with read() would crash most systems.
A Worked Example
Say you're processing a sales report saved as report.txt. The file has product names in the first column and prices in the second, with one item per line.
You need the total revenue. Here's how you'd handle it:
“`python
With open('report.txt', 'r') as f:
Next(f) # skip header line
Total = 0
For line in f:
Parts = line.strip().split(',')
If len(parts) >= 2:
Try:
Total += float(parts[1])
Except ValueError:
Continue
Print(f"Total revenue: {total}")
“`
The iteration keeps memory usage flat regardless of file size. The try/except handles malformed lines gracefully.
What to Watch Next
Once you're comfortable with basic file reading, explore the csv module for structured data. It handles quoting, escaping, and edge cases that manual parsing misses.
For JSON files, use the json module. For large datasets, pandas provides powerful tools that build on these fundamentals.
The patterns here extend to network sockets, HTTP responses, and compressed archives. Master text files and you'll find other reading tasks feel familiar.
Key takeaways
- Use with open() for automatic file cleanup and error-safe handling
- Read() loads everything; iterate line-by-line for large files
- Choose 'r' for reading, 'w' for writing, 'a' for appending
- Specify encoding='utf-8' to avoid cross-platform text problems
- Try/except blocks handle missing files and malformed data gracefully
Frequently asked questions
What happens if I open a file in read mode that doesn't exist?
Python raises a FileNotFoundError exception.
Should I specify the encoding when opening files?
Yes, always use encoding='utf-8'. Relying on system defaults causes bugs when files contain special characters.
How do I read only the first few lines of a file?
Use a for loop with a counter and break after reaching your desired line count.
Can I read files over the internet instead of from disk?
Yes. The urllib.request module provides file-like objects that work with the same read() methods.
Why does my text file look correct but Python reads extra blank lines?
The print() function adds its own newline. Use line.strip() to remove the line terminator before printing.