![\"file\"](\"\/wp-content\/uploads\/2025\/09\/image-1757816302068.png\")
<\/p>\nSo far, we have learned how to write programs and communicate our intentions to the Central Processing Unit using conditional execution, funtions, and iterations. We have learned how to create and use data structures in the Main Memory. The CPU and memory are where our software works and runs. It is where all of the "thinking" happens.<\/p>\n
But if you recall from hardware architecture discussions, once the power is turned off, anything stored in either the CPU or main memory is erased. So up to now, our programs have just been transient fun exercises to learn Python.<\/p>\n
<\/p>\n
Secondary Memory
\nIn this chapter, we start to work with Secondary Memory (or files). Secondary Memory is not erased when the power is turned off. Or in the case of a USB flash drive, the data we write from our programs can be removed from the system and transported to another system.<\/p>\n
We will primarily focus on reading and writing text files such as thoes we create in a text editor. Later we will see how to work with database files which are binary files, specifically designed to be read and written through database software.<\/p>\n
When we want to read or write a file (say on your hard drive), we first must open the file. Opening the file communicates with your opening system, which knows where the data for each file is stored. When you open a file, you are asking the operating system to find the file by name and make sure the file exists. In this example, we open the file mbox.txt, which should be stored in the same folder that you are in when you start Python. You can download this file from www.py4e.com\/code3\/mbox.txt<\/a><\/p>\n If the A File Handle Later we will use A text file can be thought of as a sequence of lines, much like a Python string can be thought of as a sequence of characters, this is a sample of a text file which records mail activity from various individuals in an open source project development team:<\/p>\n The entire file of mail interactions is available from<\/p>\n www.py4e.com\/code3\/mbox.txt<\/a><\/p>\n and a shortened version of the file is available from<\/p>\n www.py4e.com\/code3\/mbox-short.txt<\/a><\/p>\n These files are in a standard format for a file containing multiple mail messages. The lines which start with "From" separate the messages and lines which start with "From:" are part of the messages. For more information about mbox format, see<\/p>\n https:\/\/en.wikipedia.org\/wiki\/Mbox<\/a><\/p>\n To break the file into lines, there is a special character that represents the "end of the line" called the newline character.<\/p>\n In Python, we represent the newline charater as a backslash-n in string constants. Even though this looks like two characters, it is actually a single character. When we look at the variable by entering "stuff" in th interpreter, it shows us the You can also see that the length of the string So when we look at the lines in a file, we need to imagine that there is a special invisible character called the newline at the end of each line that marks the end of the line.<\/p>\n So the newline character separates the characters in the file into lines.<\/p>\n While the file handle does not contain the data for the file, it is quite easy to construct a We can use the file handle as the sequence in our The reason that the When the file is read using a Because the If you know the file is relatively small compared to the size of your main memory, you can read the whole file into one string using the In this example, the entire contents (all 94626 characters) of the file mbox-short.txt are read directly into the variable When the file is read in this manner, all the characters including all of the lines and newline characters are one big string in the variable Remember that this form of the When you are searching through data in a file, it is a very common patterns to read through a file, ignoring most of the lines and only processing lines which meet a particular condition. We can combine the pattern for reading a file with string method to build simple search mechanisms.<\/p>\n For example, if we wanted to read a file and only print out lines which started with the prefix "From:", we could use the string method startwith to select only those lines with the desired prefix:<\/p>\n When this program runs, we get the following output:<\/p>\n The output looks great since the only lines we are seeing are those which start with "From:", but why are we seeing the extra blank lines? This is due to that invisible newline character. Each of the lines ends with a newline, so the We would use line slicing to print all but the last character, but a simpler approach is to use the rstrip method which strips whitespaces from the right side of a string as follows:<\/p>\n When this program runs, we get the following output:<\/p>\n As your file processing programs get more complicated, you may want to structure your search loops using We can structure the loop to follow the pattern of skipping unintersting lines as follows:<\/p>\n The output of the program is the same. In English, the uninteresting lines are those which do not start with "From:", which we skip using We can use the Which produces the following output:<\/p>\n Here we also use the contracted form of the We really do not want to have to edit our Python code every time we want to process a different file. It would be more usable to enter the file name string each time the program runs so they can use our program on different files without changing the Python code.<\/p>\n This is quite simple to do by reading the file name from the user using We read the file name from the user and place it in a variable named Before peeking at the next section, take a look at the above program and ask yourself, "What could go possibly wrong here?" or "What might our friendly user do that would cause our nice little program to ungracefully exit with a traceback, making us look not-so-cool in the eyes of our users?"<\/p>\n Using I told you not to peek. This is your last chance.<\/p>\n What if our user types something that is not a file name?<\/p>\n Do not laugh. Users will eventually do every possible thing they can do to break your prorams, either mistakeenly or with malicious intent. As a matter of fact, an important part of any software development team is a person or group called Quality Assurance (or QA for short) whoese very job it is to do the craziset things possible in an attempt to break the software that the programer has created.<\/p>\n The QA team is responsible for finding the flaws in programs before we have deliverd the program to the end users who may be purchasing the software or paying our salary to write the software. So the QA team is the programmer's best friend.<\/p>\n So now that we see the flaw in the program, we can elegantly fix it using the The Protecting the Once you become more skilled in Python, you can engage in repartee with other Python programmers to decide wich of two equivalent solutions to a problem is "more Pythonic". The goal to be "more Pythonic" captures the notion that programmer is part engineering and part art. We are not always interested in just making something work, we also want our solution to be elegant and to be appriciented as elegant by our peers.<\/p>\n To write a file, you have to open it with mode "w" as a second parameter:<\/p>\n If the file already exists, opening it in write mode clears out the old data and starts fresh, so be careful! If the file doest exist, a new one is created.<\/p>\n The Again, the file object keeps track of where it is, so if you call We must make sure to manage the ends of lines as we write to the file by explicitly inserting the newline character when we want to end a line. The when you are done writing, you have to close the file to make sure that the last bit of data is physically written to the disk so it will not be lost if the power goes off.<\/p>\n we could close the files wich we open for read as well, but we can be a little sloppy if we are only opening a few files since Python makes sure that all open files are close when the program ends. when we are writing files, we want to explicitlly close the files so as to leave nothing to chance.<\/p>\n When you are reading and writing files, you might run into problems with whitespace. These errors can be hard to debug because spaces, tabs, and newlines are normally invisible:<\/p>\n The built-in function This can be helpful for debugging.<\/p>\n One other problem you might run into is that different systems use different characters to indicate the end of a line. Some systems use a newline, represented For most systems, there are applications to convert from one form to another. You can find them (and read more about this issue) at https:\/\/www.wikipedia.org\/wiki\/Newline<\/a>. Or, of couse, you could write one yourself.<\/p>\n catch<\/strong> newline<\/strong> Python<\/strong> Quality Assurance<\/strong> text file<\/strong> Exercise 1:<\/strong> Write a program to read through a file and print the contents of the file (line by line) all in upper case. Executing the program will look as follows:<\/p>\n>>> fhand = open('mbox.txt')\n>>> print(fhand)\n<_io.TextIOWrapper name='mbox.txt' mode='r' encoding='cp1252'><\/code><\/pre>\nopen<\/code> is successful, the operating system returns us a file handle. The file handle is not the actual data contained in the file, bu instead it is a "handle" that we can use to read the data. You are given a handle if the requested file exists and you have the proper permissions to read the file.<\/p>\n![\"file\"](\"\/wp-content\/uploads\/2025\/09\/image-1757818092739.png\")
<\/p>\n
\nIf the file does not exist, open<\/code> will fail with a traceback and you will not get a handle to access the contents of the file:<\/p>\n>>> fhand = open('stuff.txt')\nTraceback (most recent call last):\nFile "<stdin>", line 1, in <module>\nFileNotFoundError: [Errno 2] No such file or directory: 'stuff.txt'<\/code><\/pre>\ntry<\/code> and except<\/code> to deal more gracefully with the situation where we attempt to open a file that does not exist.<\/p>\nText files and lines<\/h3>\n
From stephen.marquard@uct.ac.za Sat Jan 5 09:14:16 2008\nReturn-Path: <postmaster@collab.sakaiproject.org>\nDate: Sat, 5 Jan 2008 09:12:18 -0500\nTo: source@collab.sakaiproject.org\nFrom: stephen.marquard@uct.ac.za\nSubject: [sakai] svn commit: r39772 - content\/branches\/\nDetails: http:\/\/source.sakaiproject.org\/viewsvn\/?view=rev&rev=39772\n...<\/code><\/pre>\n\\n<\/code> in the string, but when we use print<\/code> to show the string, we see the string broken into two lines by the newline character.<\/p>\n>>> stuff = 'Hello\\nWorld!'\n>>> stuff\n'Hello\\nWorld!'\n>>> print(stuff)\nHello\nWorld!\n>>> stuff = 'X\\nY'\n>>> print(stuff)\nX\nY\n>>> len(stuff)\n3<\/code><\/pre>\nX\\nY<\/code> is three characters because the newline character is a single charater.<\/p>\nReading files<\/h3>\n
for<\/code> loop to read through and count each of the lines in a file:<\/p>\nfhand = open('mbox-short.txt')\ncount = 0\nfor line in fhand:\n count = count + 1\nprint('Line Count:', count)\nLine Count: 2061<\/code><\/pre>\nfor<\/code> loop. Our for<\/code> loop simply counts the number of lines in the file and prints them out. The rough translation of the for<\/code> loop into English is, "for each line in the file represented by the file handle, add one to the count<\/code> variable."<\/p>\nopen<\/code> function does not read the entire file is that the file might be quite large with many gigabytes of data. The open<\/code> statement takes the same amount of time regardless of the size of the file. The for<\/code> loop actually causes the data to be read from the file.<\/p>\nfor<\/code> loop in this manner, Python takes care of splitting the data in the file into separate lines using the newline character. Python reads each line through the newline and includes the newline as the last character in the line<\/code> variable for each iteration of the for<\/code> loop.<\/p>\nfor<\/code> loop reads the data one line at a time, it can efficiently read and count the lines in very large files without running out of main memory to store the data. The above program can count the lines in any size file using very little memory since each line is read, counted, and then discarded.<\/p>\nread<\/code> method on the file handle<\/p>\n>>> fhand = open('mbox-short.txt')\n>>> inp = fhand.read()\n>>> print(len(inp))\n94626\n>>> print(inp[:20])\nFrom stephen.marquar<\/code><\/pre>\ninp<\/code>. We use string slicing to print out the first 20 characters of the string data stored in inp<\/code>.<\/p>\ninp<\/code>. It is a good idea to store the output of read<\/code> as a variable because each call to read<\/code> exhausts the resource:<\/p>\n>>> fhand = open('mbox-short.txt')\n>>> print(len(fhand.read()))\n94626\n>>> print(len(fhand.read()))\n0<\/code><\/pre>\nopen<\/code> function should only be used if the file data will fit comfortably in the main memory of your computer. If the file is too large to fit in main memory, you should write your program to read the file in chunks using a for<\/code> or while<\/code> loop.<\/p>\nSearching through a file<\/h3>\n
fhand = open('mbox-short.txt')\nfor line in fhand:\n if line.startswith('From:'):\n print(line)<\/code><\/pre>\nFrom: stephen.marquard@uct.ac.za\n\nFrom: louis@media.berkeley.edu\n\nFrom: zqian@umich.edu\n\nFrom: rjlowe@iupui.edu\n...<\/code><\/pre>\nprint<\/code> statement prints the string in the variable line which includes a newline and then print<\/code> adds another newline, resulting in the double spacing effect we see.<\/p>\nfhand = open('mbox-short.txt')\nfor line in fhand:\n line = line.rstrip()\n if line.startswith('From:'):\n print(line)<\/code><\/pre>\nFrom: stephen.marquard@uct.ac.za\nFrom: louis@media.berkeley.edu\nFrom: zqian@umich.edu\nFrom: rjlowe@iupui.edu\nFrom: zqian@umich.edu\nFrom: rjlowe@iupui.edu\nFrom: cwen@iupui.edu\n...<\/code><\/pre>\ncontinue<\/code>. The basic idea of the search loop is that you are looking for "intersting" lines and effectively skipping "unintersting" lines. And then when we find an intersting line, we do something with that line.<\/p>\nfhand = open('mbox-short.txt')\nfor line in fhand:\n line = line.rstrip()\n # Skip 'uninteresting lines'\n if not line.startswith('From:'):\n continue\n # Process our 'interesting' line\n print(line)<\/code><\/pre>\ncontinue<\/code>. For the "interesting" lines (i.e., those that start with "From:") we perform the processing.<\/p>\nfind<\/code> string method to simulate a text editor search that finds lines where the search string is anywhere in the line. Since find<\/code> looks for an occurrence of a string with another string and either returns the position of the string or -1 if the string was not found, we can write the following loop to show lines which contain the string "@uct.ac.za" (i.e., they come from the University of Cape Town in South Africa):<\/p>\nfhand = open('mbox-short.txt')\nfor line in fhand:\n line = line.rstrip()\n if line.find('@uct.ac.za') == -1: continue\n print(line)<\/code><\/pre>\nFrom stephen.marquard@uct.ac.za Sat Jan 5 09:14:16 2008\nX-Authentication-Warning: set sender to stephen.marquard@uct.ac.za using -f\nFrom: stephen.marquard@uct.ac.za\nAuthor: stephen.marquard@uct.ac.za\nFrom david.horwitz@uct.ac.za Fri Jan 4 07:02:32 2008\nX-Authentication-Warning: set sender to david.horwitz@uct.ac.za using -f\nFrom: david.horwitz@uct.ac.za\nAuthor: david.horwitz@uct.ac.za\n...<\/code><\/pre>\nif<\/code> statement where we put the continue<\/code> on the same line as the if<\/code>. This contracted form of the if<\/code> functions the same as if the continue<\/code> were on the next line and indented.<\/p>\nLetting the user choose the file name<\/h3>\n
input<\/code> as follows:<\/p>\nfname = input('Enter the file name: ')\nfhand = open(fname)\ncount = 0\nfor line in fhand:\n if line.startswith('Subject:'):\n count = count + 1\nprint('There were', count, 'subject lines in', fname)<\/code><\/pre>\nfname<\/code> and open that file. Now we can run the program repeatedly on different files.<\/p>\nEnter the file name: mbox.txt\nThere were 1797 subject lines in mbox.txt\n\nEnter the file name: mbox-short.txt\nThere were 27 subject lines in mbox-short.txt<\/code><\/pre>\ntry, except,<\/code> and open<\/code><\/p>\nEnter the file name: missing.txt\nTraceback (most recent call last):\n File "search6.py", line 2, in <module>\n fhand = open(fname)\nFileNotFoundError: [Errno 2] No such file or directory: 'missing.txt'\n\nEnter the file name: na na boo boo\nTraceback (most recent call last):\n File "search6.py", line 2, in <module>\n fhand = open(fname)\nFileNotFoundError: [Errno 2] No such file or directory: 'na na boo boo'<\/code><\/pre>\ntry<\/code>\/except<\/code> structure. We need to assume that the open<\/code> call might fail and add recovery code when the open<\/code> fails as follows:<\/p>\nfname = input('Enter the file name: ')\ntry:\n fhand = open(fname)\nexcept:\n print('File cannot be opened:', fname)\n exit()\ncount = 0\nfor line in fhand:\n if line.startswith('Subject:'):\n count = count + 1\nprint('There were', count, 'subject lines in', fname)<\/code><\/pre>\nexit<\/code> function terminates the program. It is a function that we call never returns. Now when our user (or QA team) types in silliness or bad file names, we "catch" them and recover gracefully:<\/p>\nEnter the file name: mbox.txt\nThere were 1797 subject lines in mbox.txt\n\nEnter the file name: na na boo boo\nFile cannot be opened: na na boo boo<\/code><\/pre>\nopen<\/code> call is a good example of the proper use of try<\/code> and except<\/code> in a Python program. We use the term "pythonic" when we are doing something the "Python way". We might say that the above exampls is the Pythonic way to open a file.<\/p>\nWriting files<\/h3>\n
>>> fout = open('output.txt', 'w')\n>>> print(fout)\n<_io.TextIOWrapper name='output.txt' mode='w' encoding='cp1252'><\/code><\/pre>\nwrite<\/code> method of the file handle object puts data into the file, returning the number of characters written. The default write mode is text for writing (and reading) strings.<\/p>\n>>> line1 = "This here's the wattle,\\n"\n>>> fout.write(line1)\n24<\/code><\/pre>\nwrite<\/code> again, it adds the new data to the end.<\/p>\nprint<\/code> statement automatically appends a newline, but the write<\/code> method does not add the newline automatically.<\/p>\n>>> line2 = 'the emblem of our land.\\n'\n>>> fout.write(line2)\n24<\/code><\/pre>\n>>> fout.close()<\/code><\/pre>\nDebugging<\/h3>\n
>>> s = '1 2\\t 3\\n 4'\n>>> print(s)\n1 2 3\n 4<\/code><\/pre>\nrepr<\/code> can help. It takes any object as an argument and returns a string representation of the object. For strings, it reprsents whitespace characters with backslash sequences:<\/p>\n>>> print(repr(s))\n'1 2\\t 3\\n 4'<\/code><\/pre>\n\\n<\/code>. Other use a return character, represented \\r<\/code>. Some use both. If you move files between different systems, these inconsistencies might cause problems.<\/p>\nGlassary<\/h3>\n
\nTo prevent an exception from terminating a program using the try<\/code> and except<\/code> statements.<\/p>\n
\nA special character used in files and string to indicate the end of a line.<\/p>\n
\nA technique that works elegantly in Python. "using try and except is the Pythonic way to recover from missing files"<\/p>\n
\nA person or team focused on insuring the overall quality of a software product. QA is often involved in testing a product and identifying problems before the product is released.<\/p>\n
\nA sequence of characters stored in permanent storage like a hard drive.<\/p>\nExercises<\/h3>\n
fname = input('Enter file name: ')\ntry:\n fhand = open(fname,encoding='utf-8')\nexcept:\n print('please enter a valid file')\n exit()\nfor line in fhand:\n print(line.upper())<\/code><\/pre>\n# generated by AI code\n# \u63d0\u793a\u7528\u6237\u8f93\u5165\u6587\u4ef6\u540d\nfname = input('Enter a file name: ')\n\n# \u5f02\u5e38\u5904\u7406\uff1a\u6355\u83b7\u6587\u4ef6\u4e0d\u5b58\u5728\u7b49\u9519\u8bef\ntry:\n # \u6253\u5f00\u6587\u4ef6\u5e76\u9010\u884c\u8bfb\u53d6\n with open(fname, 'r') as fhand:\n # \u904d\u5386\u6587\u4ef6\u7684\u6bcf\u4e00\u884c\n for line in fhand:\n # \u53bb\u9664\u884c\u5c3e\u6362\u884c\u7b26\uff0c\u8f6c\u6362\u4e3a\u5927\u5199\u540e\u6253\u5370\n line_upper = line.rstrip().upper()\n print(line_upper)\n# \u5904\u7406\u6587\u4ef6\u65e0\u6cd5\u6253\u5f00\u7684\u60c5\u51b5\nexcept:\n print(f'Error: Cannot open file "{fname}"')\n exit()<\/code><\/pre>\n