1.. _tut-io: 2 3**************** 4Input and Output 5**************** 6 7There are several ways to present the output of a program; data can be printed 8in a human-readable form, or written to a file for future use. This chapter will 9discuss some of the possibilities. 10 11 12.. _tut-formatting: 13 14Fancier Output Formatting 15========================= 16 17So far we've encountered two ways of writing values: *expression statements* and 18the :func:`print` function. (A third way is using the :meth:`write` method 19of file objects; the standard output file can be referenced as ``sys.stdout``. 20See the Library Reference for more information on this.) 21 22Often you'll want more control over the formatting of your output than simply 23printing space-separated values. There are several ways to format output. 24 25* To use :ref:`formatted string literals <tut-f-strings>`, begin a string 26 with ``f`` or ``F`` before the opening quotation mark or triple quotation mark. 27 Inside this string, you can write a Python expression between ``{`` and ``}`` 28 characters that can refer to variables or literal values. 29 30 :: 31 32 >>> year = 2016 33 >>> event = 'Referendum' 34 >>> f'Results of the {year} {event}' 35 'Results of the 2016 Referendum' 36 37* The :meth:`str.format` method of strings requires more manual 38 effort. You'll still use ``{`` and ``}`` to mark where a variable 39 will be substituted and can provide detailed formatting directives, 40 but you'll also need to provide the information to be formatted. 41 42 :: 43 44 >>> yes_votes = 42_572_654 45 >>> no_votes = 43_132_495 46 >>> percentage = yes_votes / (yes_votes + no_votes) 47 >>> '{:-9} YES votes {:2.2%}'.format(yes_votes, percentage) 48 ' 42572654 YES votes 49.67%' 49 50* Finally, you can do all the string handling yourself by using string slicing and 51 concatenation operations to create any layout you can imagine. The 52 string type has some methods that perform useful operations for padding 53 strings to a given column width. 54 55When you don't need fancy output but just want a quick display of some 56variables for debugging purposes, you can convert any value to a string with 57the :func:`repr` or :func:`str` functions. 58 59The :func:`str` function is meant to return representations of values which are 60fairly human-readable, while :func:`repr` is meant to generate representations 61which can be read by the interpreter (or will force a :exc:`SyntaxError` if 62there is no equivalent syntax). For objects which don't have a particular 63representation for human consumption, :func:`str` will return the same value as 64:func:`repr`. Many values, such as numbers or structures like lists and 65dictionaries, have the same representation using either function. Strings, in 66particular, have two distinct representations. 67 68Some examples:: 69 70 >>> s = 'Hello, world.' 71 >>> str(s) 72 'Hello, world.' 73 >>> repr(s) 74 "'Hello, world.'" 75 >>> str(1/7) 76 '0.14285714285714285' 77 >>> x = 10 * 3.25 78 >>> y = 200 * 200 79 >>> s = 'The value of x is ' + repr(x) + ', and y is ' + repr(y) + '...' 80 >>> print(s) 81 The value of x is 32.5, and y is 40000... 82 >>> # The repr() of a string adds string quotes and backslashes: 83 ... hello = 'hello, world\n' 84 >>> hellos = repr(hello) 85 >>> print(hellos) 86 'hello, world\n' 87 >>> # The argument to repr() may be any Python object: 88 ... repr((x, y, ('spam', 'eggs'))) 89 "(32.5, 40000, ('spam', 'eggs'))" 90 91The :mod:`string` module contains a :class:`~string.Template` class that offers 92yet another way to substitute values into strings, using placeholders like 93``$x`` and replacing them with values from a dictionary, but offers much less 94control of the formatting. 95 96 97.. _tut-f-strings: 98 99Formatted String Literals 100------------------------- 101 102:ref:`Formatted string literals <f-strings>` (also called f-strings for 103short) let you include the value of Python expressions inside a string by 104prefixing the string with ``f`` or ``F`` and writing expressions as 105``{expression}``. 106 107An optional format specifier can follow the expression. This allows greater 108control over how the value is formatted. The following example rounds pi to 109three places after the decimal:: 110 111 >>> import math 112 >>> print(f'The value of pi is approximately {math.pi:.3f}.') 113 The value of pi is approximately 3.142. 114 115Passing an integer after the ``':'`` will cause that field to be a minimum 116number of characters wide. This is useful for making columns line up. :: 117 118 >>> table = {'Sjoerd': 4127, 'Jack': 4098, 'Dcab': 7678} 119 >>> for name, phone in table.items(): 120 ... print(f'{name:10} ==> {phone:10d}') 121 ... 122 Sjoerd ==> 4127 123 Jack ==> 4098 124 Dcab ==> 7678 125 126Other modifiers can be used to convert the value before it is formatted. 127``'!a'`` applies :func:`ascii`, ``'!s'`` applies :func:`str`, and ``'!r'`` 128applies :func:`repr`:: 129 130 >>> animals = 'eels' 131 >>> print(f'My hovercraft is full of {animals}.') 132 My hovercraft is full of eels. 133 >>> print(f'My hovercraft is full of {animals!r}.') 134 My hovercraft is full of 'eels'. 135 136The ``=`` specifier can be used to expand an expression to the text of the 137expression, an equal sign, then the representation of the evaluated expression: 138 139 >>> bugs = 'roaches' 140 >>> count = 13 141 >>> area = 'living room' 142 >>> print(f'Debugging {bugs=} {count=} {area=}') 143 Debugging bugs='roaches' count=13 area='living room' 144 145See :ref:`self-documenting expressions <bpo-36817-whatsnew>` for more information 146on the ``=`` specifier. For a reference on these format specifications, see 147the reference guide for the :ref:`formatspec`. 148 149.. _tut-string-format: 150 151The String format() Method 152-------------------------- 153 154Basic usage of the :meth:`str.format` method looks like this:: 155 156 >>> print('We are the {} who say "{}!"'.format('knights', 'Ni')) 157 We are the knights who say "Ni!" 158 159The brackets and characters within them (called format fields) are replaced with 160the objects passed into the :meth:`str.format` method. A number in the 161brackets can be used to refer to the position of the object passed into the 162:meth:`str.format` method. :: 163 164 >>> print('{0} and {1}'.format('spam', 'eggs')) 165 spam and eggs 166 >>> print('{1} and {0}'.format('spam', 'eggs')) 167 eggs and spam 168 169If keyword arguments are used in the :meth:`str.format` method, their values 170are referred to by using the name of the argument. :: 171 172 >>> print('This {food} is {adjective}.'.format( 173 ... food='spam', adjective='absolutely horrible')) 174 This spam is absolutely horrible. 175 176Positional and keyword arguments can be arbitrarily combined:: 177 178 >>> print('The story of {0}, {1}, and {other}.'.format('Bill', 'Manfred', 179 ... other='Georg')) 180 The story of Bill, Manfred, and Georg. 181 182If you have a really long format string that you don't want to split up, it 183would be nice if you could reference the variables to be formatted by name 184instead of by position. This can be done by simply passing the dict and using 185square brackets ``'[]'`` to access the keys. :: 186 187 >>> table = {'Sjoerd': 4127, 'Jack': 4098, 'Dcab': 8637678} 188 >>> print('Jack: {0[Jack]:d}; Sjoerd: {0[Sjoerd]:d}; ' 189 ... 'Dcab: {0[Dcab]:d}'.format(table)) 190 Jack: 4098; Sjoerd: 4127; Dcab: 8637678 191 192This could also be done by passing the ``table`` dictionary as keyword arguments with the ``**`` 193notation. :: 194 195 >>> table = {'Sjoerd': 4127, 'Jack': 4098, 'Dcab': 8637678} 196 >>> print('Jack: {Jack:d}; Sjoerd: {Sjoerd:d}; Dcab: {Dcab:d}'.format(**table)) 197 Jack: 4098; Sjoerd: 4127; Dcab: 8637678 198 199This is particularly useful in combination with the built-in function 200:func:`vars`, which returns a dictionary containing all local variables. 201 202As an example, the following lines produce a tidily aligned 203set of columns giving integers and their squares and cubes:: 204 205 >>> for x in range(1, 11): 206 ... print('{0:2d} {1:3d} {2:4d}'.format(x, x*x, x*x*x)) 207 ... 208 1 1 1 209 2 4 8 210 3 9 27 211 4 16 64 212 5 25 125 213 6 36 216 214 7 49 343 215 8 64 512 216 9 81 729 217 10 100 1000 218 219For a complete overview of string formatting with :meth:`str.format`, see 220:ref:`formatstrings`. 221 222 223Manual String Formatting 224------------------------ 225 226Here's the same table of squares and cubes, formatted manually:: 227 228 >>> for x in range(1, 11): 229 ... print(repr(x).rjust(2), repr(x*x).rjust(3), end=' ') 230 ... # Note use of 'end' on previous line 231 ... print(repr(x*x*x).rjust(4)) 232 ... 233 1 1 1 234 2 4 8 235 3 9 27 236 4 16 64 237 5 25 125 238 6 36 216 239 7 49 343 240 8 64 512 241 9 81 729 242 10 100 1000 243 244(Note that the one space between each column was added by the 245way :func:`print` works: it always adds spaces between its arguments.) 246 247The :meth:`str.rjust` method of string objects right-justifies a string in a 248field of a given width by padding it with spaces on the left. There are 249similar methods :meth:`str.ljust` and :meth:`str.center`. These methods do 250not write anything, they just return a new string. If the input string is too 251long, they don't truncate it, but return it unchanged; this will mess up your 252column lay-out but that's usually better than the alternative, which would be 253lying about a value. (If you really want truncation you can always add a 254slice operation, as in ``x.ljust(n)[:n]``.) 255 256There is another method, :meth:`str.zfill`, which pads a numeric string on the 257left with zeros. It understands about plus and minus signs:: 258 259 >>> '12'.zfill(5) 260 '00012' 261 >>> '-3.14'.zfill(7) 262 '-003.14' 263 >>> '3.14159265359'.zfill(5) 264 '3.14159265359' 265 266 267Old string formatting 268--------------------- 269 270The % operator (modulo) can also be used for string formatting. Given ``'string' 271% values``, instances of ``%`` in ``string`` are replaced with zero or more 272elements of ``values``. This operation is commonly known as string 273interpolation. For example:: 274 275 >>> import math 276 >>> print('The value of pi is approximately %5.3f.' % math.pi) 277 The value of pi is approximately 3.142. 278 279More information can be found in the :ref:`old-string-formatting` section. 280 281 282.. _tut-files: 283 284Reading and Writing Files 285========================= 286 287.. index:: 288 pair: built-in function; open 289 pair: object; file 290 291:func:`open` returns a :term:`file object`, and is most commonly used with 292two positional arguments and one keyword argument: 293``open(filename, mode, encoding=None)`` 294 295:: 296 297 >>> f = open('workfile', 'w', encoding="utf-8") 298 299.. XXX str(f) is <io.TextIOWrapper object at 0x82e8dc4> 300 301 >>> print(f) 302 <open file 'workfile', mode 'w' at 80a0960> 303 304The first argument is a string containing the filename. The second argument is 305another string containing a few characters describing the way in which the file 306will be used. *mode* can be ``'r'`` when the file will only be read, ``'w'`` 307for only writing (an existing file with the same name will be erased), and 308``'a'`` opens the file for appending; any data written to the file is 309automatically added to the end. ``'r+'`` opens the file for both reading and 310writing. The *mode* argument is optional; ``'r'`` will be assumed if it's 311omitted. 312 313Normally, files are opened in :dfn:`text mode`, that means, you read and write 314strings from and to the file, which are encoded in a specific *encoding*. 315If *encoding* is not specified, the default is platform dependent 316(see :func:`open`). 317Because UTF-8 is the modern de-facto standard, ``encoding="utf-8"`` is 318recommended unless you know that you need to use a different encoding. 319Appending a ``'b'`` to the mode opens the file in :dfn:`binary mode`. 320Binary mode data is read and written as :class:`bytes` objects. 321You can not specify *encoding* when opening file in binary mode. 322 323In text mode, the default when reading is to convert platform-specific line 324endings (``\n`` on Unix, ``\r\n`` on Windows) to just ``\n``. When writing in 325text mode, the default is to convert occurrences of ``\n`` back to 326platform-specific line endings. This behind-the-scenes modification 327to file data is fine for text files, but will corrupt binary data like that in 328:file:`JPEG` or :file:`EXE` files. Be very careful to use binary mode when 329reading and writing such files. 330 331It is good practice to use the :keyword:`with` keyword when dealing 332with file objects. The advantage is that the file is properly closed 333after its suite finishes, even if an exception is raised at some 334point. Using :keyword:`!with` is also much shorter than writing 335equivalent :keyword:`try`\ -\ :keyword:`finally` blocks:: 336 337 >>> with open('workfile', encoding="utf-8") as f: 338 ... read_data = f.read() 339 340 >>> # We can check that the file has been automatically closed. 341 >>> f.closed 342 True 343 344If you're not using the :keyword:`with` keyword, then you should call 345``f.close()`` to close the file and immediately free up any system 346resources used by it. 347 348.. warning:: 349 Calling ``f.write()`` without using the :keyword:`!with` keyword or calling 350 ``f.close()`` **might** result in the arguments 351 of ``f.write()`` not being completely written to the disk, even if the 352 program exits successfully. 353 354.. 355 See also https://bugs.python.org/issue17852 356 357After a file object is closed, either by a :keyword:`with` statement 358or by calling ``f.close()``, attempts to use the file object will 359automatically fail. :: 360 361 >>> f.close() 362 >>> f.read() 363 Traceback (most recent call last): 364 File "<stdin>", line 1, in <module> 365 ValueError: I/O operation on closed file. 366 367 368.. _tut-filemethods: 369 370Methods of File Objects 371----------------------- 372 373The rest of the examples in this section will assume that a file object called 374``f`` has already been created. 375 376To read a file's contents, call ``f.read(size)``, which reads some quantity of 377data and returns it as a string (in text mode) or bytes object (in binary mode). 378*size* is an optional numeric argument. When *size* is omitted or negative, the 379entire contents of the file will be read and returned; it's your problem if the 380file is twice as large as your machine's memory. Otherwise, at most *size* 381characters (in text mode) or *size* bytes (in binary mode) are read and returned. 382If the end of the file has been reached, ``f.read()`` will return an empty 383string (``''``). :: 384 385 >>> f.read() 386 'This is the entire file.\n' 387 >>> f.read() 388 '' 389 390``f.readline()`` reads a single line from the file; a newline character (``\n``) 391is left at the end of the string, and is only omitted on the last line of the 392file if the file doesn't end in a newline. This makes the return value 393unambiguous; if ``f.readline()`` returns an empty string, the end of the file 394has been reached, while a blank line is represented by ``'\n'``, a string 395containing only a single newline. :: 396 397 >>> f.readline() 398 'This is the first line of the file.\n' 399 >>> f.readline() 400 'Second line of the file\n' 401 >>> f.readline() 402 '' 403 404For reading lines from a file, you can loop over the file object. This is memory 405efficient, fast, and leads to simple code:: 406 407 >>> for line in f: 408 ... print(line, end='') 409 ... 410 This is the first line of the file. 411 Second line of the file 412 413If you want to read all the lines of a file in a list you can also use 414``list(f)`` or ``f.readlines()``. 415 416``f.write(string)`` writes the contents of *string* to the file, returning 417the number of characters written. :: 418 419 >>> f.write('This is a test\n') 420 15 421 422Other types of objects need to be converted -- either to a string (in text mode) 423or a bytes object (in binary mode) -- before writing them:: 424 425 >>> value = ('the answer', 42) 426 >>> s = str(value) # convert the tuple to string 427 >>> f.write(s) 428 18 429 430``f.tell()`` returns an integer giving the file object's current position in the file 431represented as number of bytes from the beginning of the file when in binary mode and 432an opaque number when in text mode. 433 434To change the file object's position, use ``f.seek(offset, whence)``. The position is computed 435from adding *offset* to a reference point; the reference point is selected by 436the *whence* argument. A *whence* value of 0 measures from the beginning 437of the file, 1 uses the current file position, and 2 uses the end of the file as 438the reference point. *whence* can be omitted and defaults to 0, using the 439beginning of the file as the reference point. :: 440 441 >>> f = open('workfile', 'rb+') 442 >>> f.write(b'0123456789abcdef') 443 16 444 >>> f.seek(5) # Go to the 6th byte in the file 445 5 446 >>> f.read(1) 447 b'5' 448 >>> f.seek(-3, 2) # Go to the 3rd byte before the end 449 13 450 >>> f.read(1) 451 b'd' 452 453In text files (those opened without a ``b`` in the mode string), only seeks 454relative to the beginning of the file are allowed (the exception being seeking 455to the very file end with ``seek(0, 2)``) and the only valid *offset* values are 456those returned from the ``f.tell()``, or zero. Any other *offset* value produces 457undefined behaviour. 458 459File objects have some additional methods, such as :meth:`~file.isatty` and 460:meth:`~file.truncate` which are less frequently used; consult the Library 461Reference for a complete guide to file objects. 462 463 464.. _tut-json: 465 466Saving structured data with :mod:`json` 467--------------------------------------- 468 469.. index:: pair: module; json 470 471Strings can easily be written to and read from a file. Numbers take a bit more 472effort, since the :meth:`read` method only returns strings, which will have to 473be passed to a function like :func:`int`, which takes a string like ``'123'`` 474and returns its numeric value 123. When you want to save more complex data 475types like nested lists and dictionaries, parsing and serializing by hand 476becomes complicated. 477 478Rather than having users constantly writing and debugging code to save 479complicated data types to files, Python allows you to use the popular data 480interchange format called `JSON (JavaScript Object Notation) 481<https://json.org>`_. The standard module called :mod:`json` can take Python 482data hierarchies, and convert them to string representations; this process is 483called :dfn:`serializing`. Reconstructing the data from the string representation 484is called :dfn:`deserializing`. Between serializing and deserializing, the 485string representing the object may have been stored in a file or data, or 486sent over a network connection to some distant machine. 487 488.. note:: 489 The JSON format is commonly used by modern applications to allow for data 490 exchange. Many programmers are already familiar with it, which makes 491 it a good choice for interoperability. 492 493If you have an object ``x``, you can view its JSON string representation with a 494simple line of code:: 495 496 >>> import json 497 >>> x = [1, 'simple', 'list'] 498 >>> json.dumps(x) 499 '[1, "simple", "list"]' 500 501Another variant of the :func:`~json.dumps` function, called :func:`~json.dump`, 502simply serializes the object to a :term:`text file`. So if ``f`` is a 503:term:`text file` object opened for writing, we can do this:: 504 505 json.dump(x, f) 506 507To decode the object again, if ``f`` is a :term:`binary file` or 508:term:`text file` object which has been opened for reading:: 509 510 x = json.load(f) 511 512.. note:: 513 JSON files must be encoded in UTF-8. Use ``encoding="utf-8"`` when opening 514 JSON file as a :term:`text file` for both of reading and writing. 515 516This simple serialization technique can handle lists and dictionaries, but 517serializing arbitrary class instances in JSON requires a bit of extra effort. 518The reference for the :mod:`json` module contains an explanation of this. 519 520.. seealso:: 521 522 :mod:`pickle` - the pickle module 523 524 Contrary to :ref:`JSON <tut-json>`, *pickle* is a protocol which allows 525 the serialization of arbitrarily complex Python objects. As such, it is 526 specific to Python and cannot be used to communicate with applications 527 written in other languages. It is also insecure by default: 528 deserializing pickle data coming from an untrusted source can execute 529 arbitrary code, if the data was crafted by a skilled attacker. 530