Chuyển tới nội dung
Home » Python Class Context Manager | Python Context Manager Protocol

Python Class Context Manager | Python Context Manager Protocol

CONTEXT MANAGERS In Python Are GENIUS!

Using the async with Statement

The

with

statement also has an asynchronous version,

async with

. You can use it to write context managers that depend on asynchronous code. It’s quite common to see

async with

in that kind of code, as many IO operations involve setup and teardown phases.

For example, say you need to code an asynchronous function to check if a given site is online. To do that, you can use

aiohttp

,

asyncio

, and

async with

like this:


1# site_checker_v0.py 2 3import aiohttp 4import asyncio 5 6async def check(url): 7 async with aiohttp.ClientSession() as session: 8 async with session.get(url) as response: 9 print(f"{url}: status -> {response.status}") 10 html = await response.text() 11 print(f"{url}: type -> {html[:17].strip()}") 12 13async def main(): 14 await asyncio.gather( 15 check("https://realpython.com"), 16 check("https://pycoders.com"), 17 ) 18 19asyncio.run(main())

Here’s what this script does:

  • Line 3 imports

    aiohttp

    , which provides an asynchronous HTTP client and server for

    asyncio

    and Python. Note that

    aiohttp

    is a third-party package that you can install by running

    python -m pip install aiohttp

    on your command line.
  • Line 4 imports

    asyncio

    , which allows you to write concurrent code using the

    async

    and

    await

    syntax.
  • Line 6 defines

    check()

    as an asynchronous function using the

    async

    keyword.

Inside

check()

, you define two nested

async with

statements:

  • Line 7 defines an outer

    async with

    that instantiates

    aiohttp.ClientSession()

    to get a context manager. It stores the returned object in

    session

    .
  • Line 8 defines an inner

    async with

    statement that calls

    .get()

    on

    session

    using

    url

    as an argument. This creates a second context manager and returns a

    response

    .
  • Line 9 prints the response status code for the

    url

    at hand.
  • Line 10 runs an awaitable call to

    .text()

    on

    response

    and stores the result in

    html

    .
  • Line 11 prints the site

    url

    and its document type,

    doctype

    .
  • Line 13 defines the script’s

    main()

    function, which is also a coroutine.
  • Line 14 calls

    gather()

    from

    asyncio

    . This function runs awaitable objects in a sequence concurrently. In this example,

    gather()

    runs two instances of

    check()

    with a different URL for each.
  • Line 19 runs

    main()

    using

    asyncio.run()

    . This function creates a new

    asyncio

    event loop and closes it at the end of the operation.

If you run this script from your command line, then you get an output similar to the following:


$ python site_checker_v0.py https://realpython.com: status -> 200 https://pycoders.com: status -> 200 https://pycoders.com: type ->

https://realpython.com: type ->

Cool! Your script works and you confirm that both sites are currently available. You also retrieve the information regarding document type from each site’s home page.

Note: Your output can look slightly different due to the nondeterministic nature of concurrent task scheduling and network latency. In particular, the individual lines can come out in a different order.

The

async with

statement works similar to the regular

with

statement, but it requires an asynchronous context manager. In other words, it needs a context manager that is able to suspend execution in its enter and exit methods. Asynchronous context managers implement the special methods

.__aenter__()

and

.__aexit__()

, which correspond to

.__enter__()

and

.__exit__()

in a regular context manager.

The

async with ctx_mgr

construct implicitly uses

await ctx_mgr.__aenter__()

when entering the context and

await ctx_mgr.__aexit__()

when exiting it. This achieves

async

context manager behavior seamlessly.

Managing Resources in Python

One common problem you’ll face in programming is how to properly manage external resources, such as files, locks, and network connections. Sometimes, a program will retain those resources forever, even if you no longer need them. This kind of issue is called a memory leak because the available memory gets reduced every time you create and open a new instance of a given resource without closing an existing one.

Managing resources properly is often a tricky problem. It requires both a setup phase and a teardown phase. The latter phase requires you to perform some cleanup actions, such as closing a file, releasing a lock, or closing a network connection. If you forget to perform these cleanup actions, then your application keeps the resource alive. This might compromise valuable system resources, such as memory and network bandwidth.

For example, a common problem that can arise when developers are working with databases is when a program keeps creating new connections without releasing or reusing them. In that case, the database back end can stop accepting new connections. This might require an admin to log in and manually kill those stale connections to make the database usable again.

Another frequent issue shows up when developers are working with files. Writing text to files is usually a buffered operation. This means that calling

.write()

on a file won’t immediately result in writing text to the physical file but to a temporary buffer. Sometimes, when the buffer isn’t full and developers forget to call

.close()

, part of the data can be lost forever.

Another possibility is that your application runs into errors or exceptions that cause the control flow to bypass the code responsible for releasing the resource at hand. Here’s an example in which you use

open()

to write some text to a file:


file = open("hello.txt", "w") file.write("Hello, World!") file.close()

This implementation doesn’t guarantee the file will be closed if an exception occurs during the

.write()

call. In this case, the code will never call

.close()

, and therefore your program might leak a file descriptor.

In Python, you can use two general approaches to deal with resource management. You can wrap your code in:

  1. A

    try



    finally

    construct
  2. A

    with

    construct

The first approach is quite general and allows you to provide setup and teardown code to manage any kind of resource. However, it’s a little bit verbose. Also, what if you forget any cleanup actions?

The second approach provides a straightforward way to provide and reuse setup and teardown code. In this case, you’ll have the limitation that the

with

statement only works with context managers. In the next two sections, you’ll learn how to use both approaches in your code.

The try … finally Approach

Working with files is probably the most common example of resource management in programming. In Python, you can use a

try



finally

statement to handle opening and closing files properly:


# Safely open the file file = open("hello.txt", "w") try: file.write("Hello, World!") finally: # Make sure to close the file after using it file.close()

In this example, you need to safely open the file

hello.txt

, which you can do by wrapping the call to

open()

in a

try



except

statement. Later, when you try to write to

file

, the

finally

clause will guarantee that

file

is properly closed, even if an exception occurs during the call to

.write()

in the

try

clause. You can use this pattern to handle setup and teardown logic when you’re managing external resources in Python.

The

try

block in the above example can potentially raise exceptions, such as

AttributeError

or

NameError

. You can handle those exceptions in an

except

clause like this:


# Safely open the file file = open("hello.txt", "w") try: file.write("Hello, World!") except Exception as e: print(f"An error occurred while writing to the file: {e}") finally: # Make sure to close the file after using it file.close()

In this example, you catch any potential exceptions that can occur while writing to the file. In real-life situations, you should use a specific exception type instead of the general

Exception

to prevent unknown errors from passing silently.

The with Statement Approach

The Python

with

statement creates a runtime context that allows you to run a group of statements under the control of a context manager. PEP 343 added the

with

statement to make it possible to factor out standard use cases of the

try



finally

statement.

Compared to traditional

try



finally

constructs, the

with

statement can make your code clearer, safer, and reusable. Many classes in the standard library support the

with

statement. A classic example of this is

open()

, which allows you to work with file objects using

with

.

To write a

with

statement, you need to use the following general syntax:


with expression as target_var: do_something(target_var)

The context manager object results from evaluating the

expression

after

with

. In other words,

expression

must return an object that implements the context management protocol. This protocol consists of two special methods:


  1. .__enter__()

    is called by the

    with

    statement to enter the runtime context.

  2. .__exit__()

    is called when the execution leaves the

    with

    code block.

The

as

specifier is optional. If you provide a

target_var

with

as

, then the return value of calling

.__enter__()

on the context manager object is bound to that variable.

Note: Some context managers return

None

from

.__enter__()

because they have no useful object to give back to the caller. In these cases, specifying a

target_var

makes no sense.

Here’s how the

with

statement proceeds when Python runs into it:

  1. Call

    expression

    to obtain a context manager.
  2. Store the context manager’s

    .__enter__()

    and

    .__exit__()

    methods for later use.
  3. Call

    .__enter__()

    on the context manager and bind its return value to

    target_var

    if provided.
  4. Execute the

    with

    code block.
  5. Call

    .__exit__()

    on the context manager when the

    with

    code block finishes.

In this case,

.__enter__()

, typically provides the setup code. The

with

statement is a compound statement that starts a code block, like a conditional statement or a

for

loop. Inside this code block, you can run several statements. Typically, you use the

with

code block to manipulate

target_var

if applicable.

Once the

with

code block finishes,

.__exit__()

gets called. This method typically provides the teardown logic or cleanup code, such as calling

.close()

on an open file object. That’s why the

with

statement is so useful. It makes properly acquiring and releasing resources a breeze.

Here’s how to open your

hello.txt

file for writing using the

with

statement:


with open("hello.txt", mode="w") as file: file.write("Hello, World!")

When you run this

with

statement,

open()

returns an

io.TextIOBase

object. This object is also a context manager, so the

with

statement calls

.__enter__()

and assigns its return value to

file

. Then you can manipulate the file inside the

with

code block. When the block ends,

.__exit__()

automatically gets called and closes the file for you, even if an exception is raised inside the

with

block.

This

with

construct is shorter than its

try



finally

alternative, but it’s also less general, as you already saw. You can only use the

with

statement with objects that support the context management protocol, whereas

try



finally

allows you to perform cleanup actions for arbitrary objects without the need for supporting the context management protocol.

In Python 3.1 and later, the

with

statement supports multiple context managers. You can supply any number of context managers separated by commas:


with A() as a, B() as b: pass

This works like nested

with

statements but without nesting. This might be useful when you need to open two files at a time, the first for reading and the second for writing:


with open("input.txt") as in_file, open("output.txt", "w") as out_file: # Read content from input.txt # Transform the content # Write the transformed content to output.txt pass

In this example, you can add code for reading and transforming the content of

input.txt

. Then you write the final result to

output.txt

in the same code block.

Using multiple context managers in a single

with

has a drawback, though. If you use this feature, then you’ll probably break your line length limit. To work around this, you need to use backslashes () for line continuation, so you might end up with an ugly final result.

The

with

statement can make the code that deals with system resources more readable, reusable, and concise, not to mention safer. It helps avoid bugs and leaks by making it almost impossible to forget cleaning up, closing, and releasing a resource after you’re done with it.

Using

with

allows you to abstract away most of the resource handling logic. Instead of having to write an explicit

try



finally

statement with setup and teardown code each time,

with

takes care of that for you and avoids repetition.

CONTEXT MANAGERS In Python Are GENIUS!
CONTEXT MANAGERS In Python Are GENIUS!

What is a Context Manager in Python?

According to the Python glossary, a context manager is —

An object which controls the environment seen in a

with

statement by defining

__enter__()

and

__exit__()

methods.

That may not be noticeably clear to you. Let me explain the concept with an example.

The

with

statement in Python lets you run a block of code within a runtime context defined by a context manager object.

Once the block of code has finished executing, the context manager object will take care of tearing down any external resources that are no longer needed.

You can rewrite the program by using the

with

statement as follows:


def main(): with open('books.txt', 'w') as my_file: my_file.write('If Tomorrow Comes by Sidney Sheldon') if __name__ == '__main__': main()

Since the

open()

function is paired with a

with

statement in this example, the function will create a context manager.

The file object will be accessible within the context of the indented code block, which means the file object doesn’t exist outside of that scope.

The

as

keyword is useful when you want to assign a target variable to a returned object. Here, the

my_file

variable is the target and will hold the file object.

You can do whatever you want within the indented block of code and don’t have to worry about closing the file.

Because once the block of code has finished executing the context manager will close the file automatically.

So, you have rewritten the entire

try...except...finally

ladder within two lines of code using the

with

statement and a context manager.

But how does that happen? How does a context manager object handle the task of setting up and closing resources?

And where are those

__enter__()

and

__exit__()

methods you read about on the Python documentation glossary?

Well, I’m so glad you asked 🙂

27.Handling Exceptions¶

We did not talk about the

type

,

value

and

traceback

arguments of the

__exit__

method. Between the 4th and 6th step, if
an exception occurs, Python passes the type, value and traceback of the
exception to the

__exit__

method. It allows the

__exit__

method
to decide how to close the file and if any further steps are required.
In our case we are not paying any attention to them.

What if our file object raises an exception? We might be trying to access a method on the file object which it does not supports. For instance:

with File(‘demo.txt’, ‘w’) as opened_file: opened_file.undefined_function(‘Hola!’)

Let’s list the steps which are taken by the

with

statement when
an error is encountered:

  1. It passes the type, value and traceback of the error to the

    __exit__

    method.
  2. It allows the

    __exit__

    method to handle the exception.
  3. If

    __exit__

    returns

    True

    then the exception was gracefully handled.
  4. If anything other than

    True

    is returned by the

    __exit__

    method then the exception is raised by the

    with

    statement.

In our case the

__exit__

method returns

None

(when no return
statement is encountered then the method returns

None

). Therefore,
the

with

statement raises the exception:

Traceback (most recent call last): File ”

“, line 2, in

AttributeError: ‘file’ object has no attribute ‘undefined_function’

Let’s try handling the exception in the

__exit__

method:

class File(object): def __init__(self, file_name, method): self.file_obj = open(file_name, method) def __enter__(self): return self.file_obj def __exit__(self, type, value, traceback): print(“Exception has been handled”) self.file_obj.close() return True with File(‘demo.txt’, ‘w’) as opened_file: opened_file.undefined_function() # Output: Exception has been handled

Our

__exit__

method returned

True

, therefore no exception was raised
by the

with

statement.

This is not the only way to implement Context Managers. There is another way and we will be looking at it in the next section.

Python Tutorial: Context Managers - Efficiently Managing Resources
Python Tutorial: Context Managers – Efficiently Managing Resources

Why use a context manager

Context managers keep our codebases much cleaner because they encapsulate administrative boilerplate and separate it from the business logic.

Additionally, context managers are structured to carry out their exit methods regardless of what happens in the code block they frame. So even if something goes wrong in the managed block, the context manager ensures the deallocations are performed and the default settings are restored.

Let’s give a solid example. Think about operating on a file without using

with

, like in the following block.

The first thing to note is that we must always close an open file. The

finally

block would perform the close even if an error occurred. If we had to do this try-except-finally logic every time we wanted to work with a file we’d have a lot of duplicate code.

Luckily, Python’s built-in

open()

is a context manager. Therefore, using a

with

statement, we can program the same logic like this:

Here,

open()

‘s enter method opens the file and returns a file object. The

as

keyword binds the returned value to , and we use to read the contents of

random.txt

. At the end of the execution of the inner code block, the exit method runs and closes the file.

We can check whether is actually closed (

with

does not define a variable scope, we can access the variables it created from outside the statement).

It’s evident from this simple example that context managers allow us to make our code cleaner and more reusable.

Python defines several other context managers in the standard library, but it also allows programmers to define context managers of their own.

In the next section, we will work on defining custom context managers. We will first work on the simple function-based implementation and later move on to the slightly more complicated class-based definitions.

Summary

A context manager, used in a

with

statement, defines a temporary context for the given set of operations. It does so by injecting code at the beginning and at the end of the code block, setting up the context at the beginning, and tearing it down at the end.

Sign in to your Python Morsels account to save your screencast settings.

Don’t have an account yet? Sign up here.

How can you create your own context manager in Python?

A context manager is an object that can be used in a

with

block to sandwich some code between an entrance action and an exit action.

File objects can be used as context managers to automatically close the file when we’re done working with it:


>>> with open("example.txt", "w") as file: ... file.write("Hello, world!") ... 13 >>> file.closed True

Context managers need a

__enter__

method and a

__exit__

method, and the

__exit__

method should accept three positional arguments:


class Example: def __enter__(self): print("enter") def __exit__(self, exc_type, exc_val, exc_tb): print("exit")

This context manager just prints

enter

when the

with

block is entered and

exit

when the

with

block is exited:


>>> with Example(): ... print("Yay Python!") ... enter Yay Python! exit

Of course, this is a somewhat silly context manager. Let’s look at a context manager that actually does something a little bit useful.

This context manager temporarily changes the value of an environment variable:


import os class set_env_var: def __init__(self, var_name, new_value): self.var_name = var_name self.new_value = new_value def __enter__(self): self.original_value = os.environ.get(self.var_name) os.environ[self.var_name] = self.new_value def __exit__(self, exc_type, exc_val, exc_tb): if self.original_value is None: del os.environ[self.var_name] else: os.environ[self.var_name] = self.original_value

The

USER

environment variable on my machine currently has the value of

Trey

:


>>> print("USER env var is", os.environ["USER"]) USER env var is trey

If we use this context manager, within its

with

block, the

USER

environment variable will have a different value:


>>> with set_env_var("USER", "akin"): ... print("USER env var is", os.environ["USER"]) ... USER env var is akin

But after the

with

block exits, the value of that environment variable resets back to its original value:


>>> print("USER env var is", os.environ["USER"]) USER env var is trey

This is all thanks to our context manager’s

__enter__

method and a

__exit__

method, which run when our context manager’s

with

block is entered and exited.


askeyword?

You’ll sometimes see context managers used with an

as

keyword (note the

as result

below):


>>> with set_env_var("USER", "akin") as result: ... print("USER env var is", os.environ["USER"]) ... print("Result from __enter__ method:", result) ...

The

as

keyword will point a given variable name to the return value from the

__enter__

method:

In our case, we always get

None

as the value of our

result

variable:


>>> with set_env_var("USER", "akin") as result: ... print("USER env var is", os.environ["USER"]) ... print("Result from __enter__ method:", result) ... USER env var is akin Result from __enter__ method: None

This is because our

__enter__

method doesn’t return anything, so it implicitly returns the default function return value of

None

.


__enter__

Let’s look at a context manager that does return something from

__enter__

.

Here we have a program called

timer.py

:


import time class Timer: def __enter__(self): self.start = time.perf_counter() return self def __exit__(self, exc_type, exc_val, exc_tb): self.stop = time.perf_counter() self.elapsed = self.stop - self.start

This context manager will time how long it took to run a particular block of code (the block of code in our

with

block).

We can use this context manager by making a

Timer

object, using

with

to run a block of code, and then checking the

elapsed

attribute on our

Timer

object:


>>> t = Timer() >>> with t: ... result = sum(range(10_000_000)) ... >>> t.elapsed 0.28711878502508625

But there’s actually an even shorter way to use this context manager.

We can make the

Timer

object and assign it to a variable, all on one line of code, using our

with

block and the

as

keyword:


>>> with Timer() as t: ... result = sum(range(10_000_000)) ... >>> t.elapsed 0.3115791230229661

This works because our context manager’s

__enter__

method returns

self

:


def __enter__(self): self.start = time.perf_counter() return self

So it’s returning the actual context manager object to us and that’s what gets assigned to the variable in our

with

block.

Since many context managers keep track of some useful state on their own object, it’s very common to see a context manager’s

__enter__

method return

self

.


__exit__

What about that

__exit__

method?


def __exit__(self, exc_type, exc_val, exc_tb): self.stop = time.perf_counter() self.elapsed = self.stop - self.start

What are those three arguments that it accepts? And does its return value matter?

If an exception occurs within a

with

block, these three arguments passed to the context manager’s

__exit__

method will be:

But if no exception occurs, those three arguments will all be

None

.

Here’s a context manager that uses all three of those arguments:


import logging class LogException: def __init__(self, logger, level=logging.ERROR, suppress=False): self.logger, self.level, self.suppress = logger, level, suppress def __enter__(self): return self def __exit__(self, exc_type, exc_val, exc_tb): if exc_type is not None: info = (exc_type, exc_val, exc_tb) self.logger.log(self.level, "Exception occurred", exc_info=info) return self.suppress return False

This context manager logs exceptions as they occur (using Python’s

logging

module).

So we can use this

LogException

context manager like this:


import logging from log_exception import LogException logging.basicConfig(level=logging.DEBUG) logger = logging.getLogger("example") with LogException(logger): result = 1 / 0 # This will cause a ZeroDivisionError print("That's the end of our program")

When an exception occurs in our code, we’ll see the exception logged to our console:


$ python3 log_example.py ERROR:example:Exception occurred Traceback (most recent call last): File "/home/trey/_/log_example.py", line 8, in

result = 1 / 0 # This will cause a ZeroDivisionError ~~^~~ ZeroDivisionError: division by zero Traceback (most recent call last): File "/home/trey/_/log_example.py", line 8, in

result = 1 / 0 # This will cause a ZeroDivisionError ~~^~~ ZeroDivisionError: division by zero


We see

ERROR

, the name of our logger (

example

),

Exception occurred

, and then the traceback.

In this example, we also a second traceback, which was printed by Python when our program crashed.

Because our program exited, it didn’t actually print out the last line in our program (

That's the end of our program

).


__exit__

If we had passed

suppress=True

to our context manager, we’ll see something different happen:


import logging from log_exception import LogException logging.basicConfig(level=logging.DEBUG) logger = logging.getLogger("example") with LogException(logger, suppress=True): result = 1 / 0 # This will cause a ZeroDivisionError print("That's the end of our program")

Now when we run our program, the exception is logged, but then our program continues onward after the

with

block:


$ python3 log_example.py ERROR:example:Exception occurred Traceback (most recent call last): File "/home/trey/_/_/log_example.py", line 8, in

result = 1 / 0 # This will cause a ZeroDivisionError ~~^~~ ZeroDivisionError: division by zero That's the end of our program

We can see

That's the end of our program

actually prints out here!

What’s going on?

So this

suppress

argument, it’s used by our context manager to suppress an exception:


import logging class LogException: ... def __exit__(self, exc_type, exc_val, exc_tb): if exc_type is not None: info = (exc_type, exc_val, exc_tb) self.logger.log(self.level, "Exception occurred", exc_info=info) return self.suppress return False

If the

__exit__

method returns something true or truthy, whatever exception was being raised will actually be suppressed.

By default,

__exit__

returns

None

, just as every function does by default.
If we return

None

, which is falsey, or

False

, or anything that’s falsey,

__exit__

won’t do anything different from its default, which is to just continue raising that exception.

But if

True

or a truthy value is returned, the exception will be suppressed.


contextmanager?

Have you ever seen a generator function that somehow made a context manager?

Python’s

contextlib

module includes a decorator which allows for creating context managers using a function syntax (instead of using the typical class syntax we saw above):


from contextlib import contextmanager import os @contextmanager def set_env_var(var_name, new_value): original_value = os.environ.get(var_name) os.environ[var_name] = new_value try: yield finally: if original_value is None: del os.environ[var_name] else: os.environ[var_name] = original_value

Interestingly, this fancy decorator still involves

__enter__

and

__exit__

under the hood: it’s just a very clever helper for creating an object that has those methods.

This

contextmanager

decorator can sometimes be very handy, though it does have limitations!
I plan to record a separate screencast (and write a separate article) on

contextlib.contextmanager

.
Python Morsels subscribers will hear about this new screencast as soon as I publish it. 😉


__enter__&


__exit__

Context managers are objects that work in a

with

block.

You can make a context manager by creating an object that has a

__enter__

method and a

__exit__

method.

Python also includes a fancy decorator for creating context managers with a function syntax, which I’ll cover in a future screencast.

Need to fill-in gaps in your Python skills?

Sign up for my Python newsletter where I share one of my favorite Python tips every week.

Need to fill-in gaps in your Python skills? I send weekly emails designed to do just that.

Summary: in this tutorial, you’ll learn about the Python context managers and how to use them effectively

Context Managers in Python Make Life Easier
Context Managers in Python Make Life Easier

Summary

  • Use Python context managers to define runtime contexts when executing in the

    with

    statement.
  • implement the

    __enter__()

    and

    __exit__()

    methods to support the context manager protocol.

When we watch youtube videos or any tutorial about data analysis, we see that the context manager is mainly used in reading or writing some texts from a file so that we don’t have to write the code for closing the file whenever our task is finished. But this is not the only use-case of this context manager. We can use context managers in more relaxed ways. For example, we can easily interchange the working directory using a context manager. We can also access a database efficiently by this. In this article, we will discuss everything about it and see some examples that will help us use them properly in your project. So what are we waiting for? Let’s get started.

This article was published as a part of the Data Science Blogathon.

A context manager is actually a resource manager in python. For example, when we read or write some content from a file, we don’t care about closing the file. If we are using one file, then this is not a big issue. But when it comes to multiple files, this will cause problems as we are not releasing the resources we are no longer using. This is also true for database access. After a database is used, we should close the database. Otherwise, it can slow down our code. So, the context manager helps us maintain the resource adequately. Below is an example of a context manager used for accessing the file.


with open(“file.txt”,”r”) as f: print(f.read())

Here we are trying to open the file in read mode using context manager and reading the first line from the file. The file will automatically close when we leave the context.

In the previous example, we used the open() function as a context manager. There are two ways to define a context manager – one is class-based and the other is function-based. Let’s see how we can create it using class.

Python Code:

For the class-based, we have to add two methods: enter () and exit(). In the enter() method, we define the things we want to do in the context. Here we open a file. In the exit() method, we have to write things that will happen after leaving the context. In this case, we will close the file after leaving the context. Now let’s see what a function-based context manager looks like.


import contextlib @contextlib.contextmanager def my_context(): # add any set up code you need yield # add any teardown code you need

For a function-based context manager, you define a function with the

@contextlib.contextmanager

decorator. Within that function, you place the code you want to execute within the context. Following that, you use the

yield

keyword to indicate that this function serves as a context manager. After

yield

, you can include your teardown code, specifying actions to take when exiting the context. This teardown code ensures proper resource cleanup and is executed when leaving the context.

We discussed the theoretical part of the context manager. Now it is time to see some examples. Let’s see how we can access the database.


import psycopg2 @contextlib.contextmanager def database(url): # set up database connection db = psycopg2.connect(url) yield db # tear down database connection db.disconnect()

You have already understood what is happening in the function. This context manager will give access to a database in its context. When we leave the context, the connection to the database is ended. You don’t have to return value whenever creating a context manager.

Now here is another example. While using Google Colab, if the data size is too large, we attach our Google Drive to the notebook and create a folder for the data. Then we download the data in that folder and want to return it to our working directory. This we can do easily with a context manager.


@contextlib.contextmanager def in _ dir(path): # save current working directory old _ dir = os.getcwd() # switch to new working directory os.chdir(path) yield # change back to previous # working directory os.chdir(old _ dir)

See the above example. We don’t return any value from the “yield” keyword. I hope you now understand how to use context manager.

Sometimes it happens that we need to open multiple files. So what should we do? Of course, we have to use multiple context managers. Here is an example of copying the content of a file.


with open(“file.txt”, “r”) as f: contents = f.read() with open(“copy.txt”, “w”) as cf: cf.write(contents)

This will work, but if we have a large file to copy, it will cause pain for us. That will make the code slower. You can also face device freezing as that process takes a lot of resources. So what should we do? If we can copy the file line by line, that doesn’t cause any problem. To do this, we have to open two files simultaneously. See the below example.


with open(“file.txt”, “r”) as f: with open(“copy.txt”, “w”) as cf: for line in f: cf.write(line)

This is called the nested context. This method is fine when you have to open two files or three files. But when the number of files is much more, the code becomes messier and you have to face difficulties while editing the code. In Python 3.10, they introduced a new method “parenthesized context” in which you can write multiple contexts in one line using brackets. No need to use looping.


with (open(“file.txt”, “r”) as f, open(“copy.txt”, “w”) as cf): for line in f: cf.write(line)

The above code is much cleaner. For me, this is the best option for working on multiple files. Now it may happen that you will prefer others. That’s totally up to you.

Using context manager in your code ensures the proper utilization of the resource. Resource utilization is not a big deal when you are working on a small project, but when it comes to large projects, like reading a very large file that we discussed previously, context manager comes in handy. In this article, we learned –

Thank you for reading this article till the last. I hope you enjoyed this article. If there is any issue, please leave a comment. And don’t forget to check out my other articles in AnalyticsVidhya.

A. A context manager in Python is an object that defines the methods __enter__() and __exit__(). It is used to set up and tear down resources, such as file handling or database connections, in a clean and controlled manner.

A. The @contextmanager decorator is used to create a context manager as a Python generator function. It simplifies the creation of context managers by yielding control to the code block defined within the generator and handling resource setup and teardown.

A. Context managers ensure proper resource management and clean-up, which is critical to prevent resource leaks and errors. They provide a convenient and safe way to work with resources like files, databases, and network connections.

A. Yes, a context manager in Python is a construct used for resource management and is crucial for ensuring that resources are acquired and released properly, even in the presence of exceptions or errors.

The media shown in this article is not owned by Analytics Vidhya and is used at the Author’s discretion.

Conclusion

The Python

with

statement is a powerful tool when it comes to managing external resources in your programs. Its use cases, however, aren’t limited to resource management. You can use the

with

statement along with existing and custom context managers to handle the setup and teardown phases of a given process or operation.

The underlying context management protocol allows you to create custom context managers and factor out the setup and teardown logic so you can reuse them in your code.

In this tutorial, you learned:

  • What the Python

    with

    statement is for and how to use it
  • What the context management protocol is
  • How to implement your own context managers

With this knowledge, you’ll write safe, concise, and expressive code. You’ll also avoid resource leaks in your programs.

Take the Quiz: Test your knowledge with our interactive “Context Managers and Python’s with Statement” quiz. Upon completion you will receive a score so you can track your learning progress over time:

Watch Now This tutorial has a related video course created by the Real Python team. Watch it together with the written tutorial to deepen your understanding: Context Managers and Python’s with Statement

Managing Resources: In any programming language, the usage of resources like file operations or database connections is very common. But these resources are limited in supply. Therefore, the main problem lies in making sure to release these resources after usage. If they are not released then it will lead to resource leakage and may cause the system to either slow down or crash. It would be very helpful if users have a mechanism for the automatic setup and teardown of resources. In Python, it can be achieved by the usage of context managers which facilitate the proper handling of resources. The most common way of performing file operations is by using the keyword as shown below:

Building A Custom Context Manager In Python: A Closer Look
Building A Custom Context Manager In Python: A Closer Look

Summarizing the with Statement’s Advantages

To summarize what you’ve learned so far, here’s an inexhaustive list of the general benefits of using the Python

with

statement in your code:

  • Makes resource management safer than its equivalent

    try



    finally

    statements
  • Encapsulates standard uses of

    try



    finally

    statements in context managers
  • Allows reusing the code that automatically manages the setup and teardown phases of a given operation
  • Helps avoid resource leaks

Using the

with

statement consistently can improve the general quality of your code and make it safer by preventing resource leak problems.

Python context manager – bạn đã thực sự hiểu?

Bài đăng này đã không được cập nhật trong 2 năm

Bài viết gốc: https://manhhomienbienthuy.github.io/2017/05/12/python-context-managers.html (đã xin phép tác giả

)

Trong Python, context manager là một phương thức cho phép bạn cấp phát và sử dụng tài nguyên một cách hiệu quả. Context manager được sử dụng rộng rãi thông qua câu lệnh

with

. Ví dụ:


with open('foo', 'w') as f: f.write('Hora! We opened this file')

Đoạn code trên mở một file, ghi dữ liệu và đóng file lại. Nếu có bất kỳ lỗi gì xảy ra, thì file cũng luôn luôn được đảm bảo là đã đóng. Đoạn code trên nếu viết mà không sử dụng context manager thì sẽ trông như dưới đây:


f = open('foo', 'w') try: f.write('Hora! We opened this file') finally: f.close()

So sánh hai cách viết này thì chúng ta đã thấy rất rõ ràng rằng, context manager cho chúng ta cách viết code ngắn gọn hơn hẳn. Lệnh

with

cho chúng ta bảo đảm rằng file luôn luôn được đóng mà không cần biết những logic xử lý bên trong.

Hẳn là các bạn đã rất quen thuộc với những đoạn code trên, đặc biệt khi bạn đã từng nghe nói đến “Idiomatic Python”. Nhưng liệu bạn đã chắc chắn hiểu được cách làm việc chính xác với file và lý do tại sao đó lại là cách đúng không? Hoặc đơn giản hơn, bạn có biết khi nào thì mình đã thao tác sai không.

Nếu câu trả lời là không, thì bài viết này chính là dành cho bạn.

Context manager thường được sử dụng để lock các tài nguyên (trường hợp mở và đóng file là một ví dụ kinh điển cho việc này).

Quản lý tài nguyên

Tính năng quan trọng nhất và cũng là phổ biến nhất của context manager là để quản lý tài nguyên một cách chính xác. Quay lại với việc đọc và ghi file ở ví dụ trên, tại sao chúng ta phải sử dụng context manager. Mỗi khi mở một file để đọc hoặc ghi, một tài nguyên của hệ thống, trong trường hợp này là file descriptor sẽ đã bị tiêu tốn để chúng ta có thể thao tác. Thật không may là tài nguyên này lại là hữu hạn. Mỗi hệ điều hành đều có giới hạn nhất định cho số lượng file có thể mở cùng một lúc.

Không tin ư, bạn hãy xem ví dụ sau:


>>> files = [] >>> for _ in range(10000): ... files.append(open('foo', 'w')) ... Traceback (most recent call last): File "

", line 2, in

OSError: [Errno 24] Too many open files: 'foo'


Ngoài lề một chút, file descriptor thực chất là một số nguyên. Khi bạn mở một file, hệ điều hành sẽ tạo ra một entry (có thể ở kernel) lưu trữ những thông tin liên quan đến file được mở. Mỗi entry sẽ được gán với 1 số nguyên (và số này sẽ là duy nhất), cho phép người dùng thông qua đó để thao tác với file.

Thực chất người dùng đang thao tác một cách gián tiếp thông qua file descriptor (và thông qua entry của kernel) với các dữ liệu thật sự được ghi ở bộ nhớ ngoài. Việc này mang lại nhiều lợi ích như có thể chia sẻ file cho nhiều tiến trình khác nhau cũng như duy trì bảo mật cho các file đó.

Thực ra, hằng ngày bạn đều đang làm việc với file descriptor mà có thể bạn cũng không nhận ra. Hệ điều hành đã gán sẵn một số file descriptor như cho bàn phím, cho màn hình, v.v… Và mọi thao tác chúng ta làm với máy tính đều thông qua các file descriptor này. Bạn nghĩ sao về việc chuyển hướng của câu lệnh Linux như thế này:


$ sort < file_list > sorted_file_list 2>&1

Tương tự như vậy, khi bạn mở một socket, một socket descriptor cũng sẽ được sử dụng.

Quay trở lại với nội dung của bài viết. Vậy điều gì xảy khi code Python của bạn mở file mà không đóng nó lại. Rất hiển nhiên, chúng ta đã mất một file descriptor mà chúng ta sẽ không bao giờ cần đến nó nữa. Điều này đồng nghĩa với việc, số file chúng ta có thể thao tác sẽ ít dần đi, do số lượng của chúng bị giới hạn. Mà việc “quên” không đóng file nhiều khi xảy ra khá thường xuyên, và tích tiểu thành đại, đến một lúc nào đó bạn không thể mở thêm file nào nữa.

Bạn có thể dùng lệnh


ulimit -n

để kiểm tra xem hệ thống của mình cho phép mở tối đa bao nhiêu file cùng lúc.

Tất nhiên là mọi vấn đề đều có thể giải quyết được. Vẫn với ví dụ trên, chúng ta có thể xử lý bằng cách đóng từng file một như sau:


>>> files = [] >>> for _ in range(10000): ... f = open('foo', 'w') ... f.close() ... files.append(f) ... >>>

Quản lý tài nguyên hiệu quả hơn

Trên đây là một cách giải quyết tuy vẫn hoạt động tốt nhưng không được thông minh cho lắm. Trong những hệ thống phức tạp hơn, rất khó để đảm bảo rằng tất cả các file đã được đóng lại khi không dùng đến nữa.

Giả sử trong quá trình thao tác, chúng ta gặp phải một exception nào đó thì phải làm thế nào đây. Bắt exception và xử lý riêng sao? Phải bắt những exception nào mới thì gọi là đủ?

Hoặc kể cả không có exception nhưng hàm đã

return

trước khi file kịp close thì sao? Trong những trường hợp phức tạp như vậy, làm thế nào để chúng ta “nhớ” phải đóng file lại. Câu trả lời là khó tới gần như không thể (thật phũ phàng).

Trong nhiều ngôn ngữ, lập trình viên cần phải sử dụng cấu trúc kiểu như

try ... except ... finally ...

để đảm bảo rằng file sẽ được đóng. Rất may mắn, Python đã nghĩ đến những khó khăn này của chúng ta và đưa cho chúng ta một phương thức dễ dàng để làm những việc đó – context manager.

Nói một cách ngắn gọn, chúng ta cần một phương thức càng đơn giản càng tốt để đảm bảo các tài nguyên được dọn dẹp cẩn thận dù có xảy ra bất cứ chuyện gì đi chăng nữa. Và context manager sẽ cung cấp cho chúng ta tính năng này:


with something_that_returns_a_context_manager() as my_resource: do_something(my_resource) ... print('done using my_resource')

Đơn giản vậy đó. Bằng cách sử dụng

with

, chúng ta sẽ đưa mọi thứ vào trong một context manager. Chúng ta gán context manager này cho một biến, và biến đó chỉ tồn tại khi block sau đó được thực thi. Điều này giống như chúng ta tạo một hàm, nó sẽ gọi một số thao tác và khi kết thúc, nó sẽ tự dọn dẹp những gì nó tạo ra.

Một số context manager hữu ích khác

Context manager thực sự rất cần thiết trong Python, và nó đã có mặt trong thư việc chuẩn. Một số context manager có thể bạn đã từng làm việc là

zipfile.ZipFiles

,

subprocess.Popen

,

tarfile.TarFile

,

telnetlib.Telnet

,

pathlib.Path

, v.v… Thậm chí,

Lock

của

threading

cũng là context manager. Trên thực tế, tất cả những tài nguyên mà chúng ta cần

close

sau khi sử dụng đều (và rất nên) là context manager.

Việc sử dụng

Lock

tương đối đặc biệt một chút. Trong trường hợp này, tài nguyên là một mutex. Sử dụng context manager sẽ phòng tránh được deadlock trong lập trình multithread nếu chúng ta sử dụng khóa mà không bao giờ mở nó. Hãy xem xét ví dụ sau:


>>> from threading import Lock >>> lock = Lock() >>> def do_something_dangerous(): ... lock.acquire() ... raise Exception('OOPS! I forgot this code could raise exceptions') ... lock.release() ... >>> try: ... do_something_dangerous() ... except: ... print('Got an exception') ... Got an exception >>> lock.acquire()

Với code trên, rõ ràng là

lock.release()

sẽ không bao giờ được gọi, và do đó, mọi tiến trình sẽ gặp deadlock và chết cứng ở đó (

lock.acquire()

sẽ không bao giờ kết thúc). Rất may mắn, với context manager, điều này có thể sửa chữa được:


>>> from threading import Lock >>> lock = Lock() >>> def do_something_dangerous(): ... with lock: ... raise Exception('oops I forgot this code could raise exceptions') ... >>> try: ... do_something_dangerous() ... except: ... print('Got an exception') ... Got an exception >>> lock.acquire() True >>> print('We can get here') We can get here >>>

Trên thực tế, không có cách nào để gây ra deadlock nếu sử dụng context manager. Và đây là điều chúng ta đang cần.

Trong phần tiếp theo, chúng ta sẽ tìm hiểu cách cài đặt một context manager, và qua đó, chúng ta sẽ hiểu hơn về cách thức một context manager hoạt động.

Cài đặt context manager như một class

Có nhiều cách khác nhau để cài đặt một context manager. Cách đơn giản nhất là cài đặt một class với hai phương thức vô cùng đặc biệt:

__enter__



__exit__

. Phương thức

__enter__

sẽ trả về tài nguyên cần quản lý (ví dụ như file đang được mở) và

__exit__

sẽ làm việc dọn dẹp hệ thống.

Hãy xem xét ví dụ sau về một context manager khi làm việc với file:


>>> class File: ... def __init__(self, file_name, method): ... self.file_obj = open(file_name, method) ... def __enter__(self): ... return self.file_obj ... def __exit__(self, type, value, traceback): ... self.file_obj.close() ... >>>

Class trên cũng như nhiều class khác, phương thức

__init__

để để khởi tạo đối tượng, trong trường hợp này là khởi tạo tên file cần mở cùng với mode (đọc/ghi) của nó. Phương thức

__enter__

mở file và trả về đối tượng file để thao tác với file đó trong khi

__exit__

chỉ đơn giản là đóng file lại.

Với hai phương thức

__enter__



__exit__

, chúng ta có thể sử dụng class này cùng với

with

:


>>> with File('foo', 'w') as f: ... f.write('Hora! We opened this file') ... 25

Phương thức

__exit__

bắt buộc phải có 3 tham số. Dưới đây là những gì thực sự xảy ra khi chúng ta gọi context manager:

  • Câu lệnh

    with

    lưu phương thức

    __exit__

    của class

    File
  • Câu lệnh này gọi phương thức

    __enter__

    của class

    File
  • Phương thức

    __enter__

    mở file và trả về object để thao tác với file đó
  • Object được trả về được truyền cho biến
  • Chúng ta thao tác với file bằng cách ghi dữ liệu

    f.write
  • Khi kết thúc block, câu lệnh

    with

    gọi phương thức

    __exit__
  • Phương thức

    __exit__

    đóng file cho chúng ta

Xử lý exception

Trong cài đặt đơn giản trên, chúng ta đã bỏ qua 3 tham số

type

,

value

,

traceback

của phương thức

__exit__

. Tuy nhiên, trong quá trình thực thi block lệnh ở trên, nếu xảy ra một exception, Python sẽ chuyển những thông tin

type

,

value



traceback

của exception này tới phương thức

__exit__

. Điều đó giúp chúng ta có thể tùy biến phương thức

__exit__

để xử lý những vấn đề có thể xảy ra trong quá trình thực thi. Trong trường hợp của chúng ta, chúng ta chỉ cần đóng file và không cần quan tâm đến exception này.

Nhưng chuyện gì sẽ xảy ra nếu bản thân file object gặp phải một exception? Ví dụ, khi chúng ta thử gọi một phương thức không tồn tại:


>>> with File('foo', 'w') as f: ... f.undefined_function('Oops! I called an unknown method') ... Traceback (most recent call last): File "

", line 2, in

AttributeError: '_io.TextIOWrapper' object has no attribute 'undefined_function'


Dưới đây là quy trình những gì đã xảy ra khi có lỗi xảy ra:


  • type

    ,

    value

    ,

    traceback

    của lỗi đó được truyền cho

    __exit__
  • Trong phương thức

    __exit__

    , chúng ta có thể tùy ý xử lý exception đó
  • Nếu

    __exit__

    trả về

    True

    thì exception đã được xử lý hoàn toàn.
  • Nếu không, exception sẽ tiếp tục được raise bởi lệnh

    with

Trong trường hợp của chúng ta, phương thức

__exit__

không trả về bất cứ thứ gì, do đó, lệnh

with

sẽ raise exception.

Chúng ta có thể tạm xử lý exception như sau:


>>> class File: ... def __init__(self, file_name, method): ... self.file_obj = open(file_name, method) ... def __enter__(self): ... return self.file_obj ... def __exit__(self, type, value, traceback): ... print("Exception has been handled") ... self.file_obj.close() ... return True ... >>> with File('foo', 'w') as f: ... f.undefined_function() ... Exception has been handled >>>

Phương thức

__exit__

trả về

True

, do đó, không có exception nào được raise bởi lệnh

with

.

Có nhiều cách để cài đặt context manager. Trên đây là một cách đơn giản và dễ hiểu nhất. Trong phần tiếp theo, chúng ta sẽ tìm hiểu thêm một số phương pháp cài đặt nữa.

Sử dụng contextlib cài đặt context manager

Context manager quả là tiện lợi và hữu ích vô cùng. Do đó, trong thư viện chuẩn của Python có hẳn module

contextlib

với rất nhiều công cụ để tạo và làm việc với context manager.

Chúng ta có thể cài đặt context manager bằng cách sử dụng decorator và generator.

contextlib

cung cấp cho chúng ta decorator

@contextmanager

để decorate các hàm generator chỉ gọi

yield

đúng một lần duy nhất. Với decorator này, tất cả những gì diễn ra trước

yield

đều được coi là thao tác của phương thức

__enter__

. Những gì dễn ra sau đó được coi là của phương thức

__exit__

.

Hãy xem xét ví dụ của chúng ta về quản lý file khi dùng

contextlib

:


>>> from contextlib import contextmanager >>> @contextmanager ... def open_file(path, mode): ... f = open(path, mode) ... yield f ... f.close() ... >>> files = [] >>> for _ in range(10000): ... with open_file('foo', 'w') as f: ... files.append(f) ... >>> for f in files: ... if not f.closed: ... print('not closed') ... >>>

Như chúng ta đã thấy, việc cài đặt context manager đã ngắn gọn hơn rất nhiều. Chúng ta chỉ cần mở file,

yield

đối tượng đó và đóng nó lại. Mọi việc còn lại sẽ do decorator

@contextmanager

đảm nhiệm.

Và ví dụ thực tế cho thấy rằng các file của chúng ta đã được quản lý tốt, tất cả chúng đã được đóng lại đầy đủ. Tuy nhiên, cách thức cài đặt tiện lợi này yêu cầu chúng ta phải có chút hiểu biết về decorator, generator cũng như lệnh

yield

. Có thể tóm tắt quá trình tạo context manager trên như sau:

  • Python tìm thấy

    yield

    , hàm này là một generator chứ không phải hàm thông thường.
  • Với decorator

    @contextmanager

    , hàm

    open_file

    sẽ được truyền là tham số cho hàm

    contextmanager

    .
  • Hàm

    contextmanager

    trả về generator được bọc trong object của

    GeneratorContextManager

    .
  • Object

    GeneratorContextManager

    được gán cho hàm

    open_file

    . Do đó, khi chúng ta gọi hàm này, thực ra chúng ta đang làm việc với object

    GeneratorContextManager

    .

Python docs còn một ví dụ khác thú vị hơn:


>>> from contextlib import contextmanager >>> @contextmanager ... def tag(name): ... print("<%s>" % name) ... yield ... print("

" % name) ... >>> with tag('h1'): ... print('foo') ...

foo

Trong tất cả các trường hợp trên, chúng ta cũng không hề xử lý exception, vì vậy, context manager của chúng ta sẽ hoạt động giống như code đầu tiên.

Một công cụ tiện lợi khác của

contextlib



ContextDecorator

. Nó cho phép chúng ta cài đặt các context manager theo kiểu class. Nhưng với việc kế thừa từ class

ContextDecorator

, bạn có thể sử dụng context manager với lệnh

with

thông thường, hoặc sử dụng nó như một decorator dùng để decorate các hàm khác. Chúng ta có thể xem xét ví dụ sau (tương tự như ví dụ tag HTML ở trên):


>>> from contextlib import ContextDecorator >>> class tag(ContextDecorator): ... def __init__(self, name): ... self.name = name ... def __enter__(self): ... print('<%s>' % self.name) ... return self ... def __exit__(self, *exc): ... print('

' % self.name) ... return False ... >>> with tag('h1'): ... print('this is not html') ...

this is not html

>>> @tag('h1') ... def content(): ... print('this is another non-html content') ... >>> content()

this is another non-html content

>>>

Kết luận

Bài viết trình bày những hiểu biết của tôi về context manager của Python, cách nó hoạt động và hỗ trợ cho chúng ta trong công việc lập trình. Như các bạn đã thấy, chúng ta có thể làm rất nhiều thứ với context manager. Mục đích cao nhất của nó thì không bao giờ thay đổi: quản lý hiệu quả các tài nguyên.

Chúng ta không chỉ có thể dùng context manager mà còn có thể tự cài đặt context manager cho riêng mình. Hãy sử dụng context manager và làm cho cuộc sống dễ chịu hơn.

All rights reserved

Context Managers¶

Context managers allow you to allocate and release resources precisely
when you want to. The most widely used example of context managers is
the

with

statement. Suppose you have two related operations which
you’d like to execute as a pair, with a block of code in between.
Context managers allow you to do specifically that. For example:

with open(‘some_file’, ‘w’) as opened_file: opened_file.write(‘Hola!’)

The above code opens the file, writes some data to it and then closes it. If an error occurs while writing the data to the file, it tries to close it. The above code is equivalent to:

file = open(‘some_file’, ‘w’) try: file.write(‘Hola!’) finally: file.close()

While comparing it to the first example we can see that a lot of
boilerplate code is eliminated just by using

with

. The main
advantage of using a

with

statement is that it makes sure our file
is closed without paying attention to how the nested block exits.

A common use case of context managers is locking and unlocking resources and closing opened files (as I have already shown you).

Let’s see how we can implement our own Context Manager. This should allow us to understand exactly what’s going on behind the scenes.

Expert Python Tutorial #6 - Context Managers
Expert Python Tutorial #6 – Context Managers

Single use, reusable and reentrant context managers¶

Most context managers are written in a way that means they can only be
used effectively in a

with

statement once. These single use
context managers must be created afresh each time they’re used –
attempting to use them a second time will trigger an exception or
otherwise not work correctly.

This common limitation means that it is generally advisable to create
context managers directly in the header of the

with

statement
where they are used (as shown in all of the usage examples above).

Files are an example of effectively single use context managers, since
the first

with

statement will close the file, preventing any
further IO operations using that file object.

Context managers created using

contextmanager()

are also single use
context managers, and will complain about the underlying generator failing
to yield if an attempt is made to use them a second time:

>>> from contextlib import contextmanager >>> @contextmanager … def singleuse(): … print(“Before”) … yield … print(“After”) … >>> cm = singleuse() >>> with cm: … pass … Before After >>> with cm: … pass … Traceback (most recent call last): … RuntimeError: generator didn’t yield

Reentrant context managers¶

More sophisticated context managers may be “reentrant”. These context
managers can not only be used in multiple

with

statements,
but may also be used inside a

with

statement that is already
using the same context manager.


threading.RLock

is an example of a reentrant context manager, as are

suppress()

,

redirect_stdout()

, and

chdir()

. Here’s a very
simple example of reentrant use:

>>> from contextlib import redirect_stdout >>> from io import StringIO >>> stream = StringIO() >>> write_to_stream = redirect_stdout(stream) >>> with write_to_stream: … print(“This is written to the stream rather than stdout”) … with write_to_stream: … print(“This is also written to the stream”) … >>> print(“This is written directly to stdout”) This is written directly to stdout >>> print(stream.getvalue()) This is written to the stream rather than stdout This is also written to the stream

Real world examples of reentrancy are more likely to involve multiple functions calling each other and hence be far more complicated than this example.

Note also that being reentrant is not the same thing as being thread safe.

redirect_stdout()

, for example, is definitely not thread safe, as it
makes a global modification to the system state by binding

sys.stdout

to a different stream.

Reusable context managers¶

Distinct from both single use and reentrant context managers are “reusable” context managers (or, to be completely explicit, “reusable, but not reentrant” context managers, since reentrant context managers are also reusable). These context managers support being used multiple times, but will fail (or otherwise not work correctly) if the specific context manager instance has already been used in a containing with statement.


threading.Lock

is an example of a reusable, but not reentrant,
context manager (for a reentrant lock, it is necessary to use

threading.RLock

instead).

Another example of a reusable, but not reentrant, context manager is

ExitStack

, as it invokes all currently registered callbacks
when leaving any with statement, regardless of where those callbacks
were added:

>>> from contextlib import ExitStack >>> stack = ExitStack() >>> with stack: … stack.callback(print, “Callback: from first context”) … print(“Leaving first context”) … Leaving first context Callback: from first context >>> with stack: … stack.callback(print, “Callback: from second context”) … print(“Leaving second context”) … Leaving second context Callback: from second context >>> with stack: … stack.callback(print, “Callback: from outer context”) … with stack: … stack.callback(print, “Callback: from inner context”) … print(“Leaving inner context”) … print(“Leaving outer context”) … Leaving inner context Callback: from inner context Callback: from outer context Leaving outer context

As the output from the example shows, reusing a single stack object across multiple with statements works correctly, but attempting to nest them will cause the stack to be cleared at the end of the innermost with statement, which is unlikely to be desirable behaviour.

Using separate

ExitStack

instances instead of reusing a single
instance avoids that problem:

>>> from contextlib import ExitStack >>> with ExitStack() as outer_stack: … outer_stack.callback(print, “Callback: from outer context”) … with ExitStack() as inner_stack: … inner_stack.callback(print, “Callback: from inner context”) … print(“Leaving inner context”) … print(“Leaving outer context”) … Leaving inner context Callback: from inner context Leaving outer context Callback: from outer context

Context Managers in Python: Using the “with” statement

Context managers are used to set up and tear down temporary contexts, establish and resolve custom settings, and acquire and release resources. The

open()

function for opening files is one of the most familiar examples of a context manager.

Context managers sandwich code blocks between two distinct pieces of logic:

  1. The enter logic – this runs right before the nested code block executes
  2. The exit logic – this runs right after the nested code block is done.

The most common way you’ll work with context managers is by using the

with

statement.

27.Implementing a Context Manager as a Class:¶

At the very least a context manager has an

__enter__

and

__exit__

method defined. Let’s make our own file-opening Context
Manager and learn the basics.

class File(object): def __init__(self, file_name, method): self.file_obj = open(file_name, method) def __enter__(self): return self.file_obj def __exit__(self, type, value, traceback): self.file_obj.close()

Just by defining

__enter__

and

__exit__

methods we can use our new class in
a

with

statement. Let’s try:

with File(‘demo.txt’, ‘w’) as opened_file: opened_file.write(‘Hola!’)

Our

__exit__

method accepts three arguments. They are required by
every

__exit__

method which is a part of a Context Manager class.
Let’s talk about what happens under-the-hood.

  1. The

    with

    statement stores the

    __exit__

    method of the

    File

    class.
  2. It calls the

    __enter__

    method of the

    File

    class.
  3. The

    __enter__

    method opens the file and returns it.
  4. The opened file handle is passed to

    opened_file

    .
  5. We write to the file using

    .write()

    .
  6. The

    with

    statement calls the stored

    __exit__

    method.
  7. The

    __exit__

    method closes the file.
LẬP TRÌNH PYTHON CƠ BẢN #24: CONTEXT MANAGER LÀ GÌ ?
LẬP TRÌNH PYTHON CƠ BẢN #24: CONTEXT MANAGER LÀ GÌ ?

Python context manager applications

As you see from the previous example, the common usage of a context manager is to open and close files automatically.

However, you can use context managers in many other cases:

1) Open – Close

If you want to open and close a resource automatically, you can use a context manager.

For example, you can open a socket and close it using a context manager.

2) Lock – release

Context managers can help you manage locks for objects more effectively. They allow you to acquire a lock and release it automatically.

3) Start – stop

Context managers also help you to work with a scenario that requires the start and stop phases.

For example, you can use a context manager to start a timer and stop it automatically.

3) Change – reset

Context managers can work with change and reset scenario.

For example, your application needs to connect to multiple data sources. And it has a default connection.

To connect to another data source:

  • First, use a context manager to change the default connection to a new one.
  • Second, work with the new connection
  • Third, reset it back to the default connection once you complete working with the new connection.

Coding Class-Based Context Managers

To implement the context management protocol and create class-based context managers, you need to add both the

.__enter__()

and the

__exit__()

special methods to your classes. The table below summarizes how these methods work, the arguments they take, and the logic you can put in them:

Method Description
This method handles the setup logic and is called when entering a new
This method handles the teardown logic and is called when the flow of execution leaves the

When the

with

statement executes, it calls

.__enter__()

on the context manager object to signal that you’re entering into a new runtime context. If you provide a target variable with the

as

specifier, then the return value of

.__enter__()

is assigned to that variable.

When the flow of execution leaves the context,

.__exit__()

is called. If no exception occurs in the

with

code block, then the three last arguments to

.__exit__()

are set to

None

. Otherwise, they hold the type, value, and traceback associated with the exception at hand.

If the

.__exit__()

method returns

True

, then any exception that occurs in the

with

block is swallowed and the execution continues at the next statement after

with

. If

.__exit__()

returns

False

, then exceptions are propagated out of the context. This is also the default behavior when the method doesn’t return anything explicitly. You can take advantage of this feature to encapsulate exception handling inside the context manager.

Writing a Sample Class-Based Context Manager

Here’s a sample class-based context manager that implements both methods,

.__enter__()

and

.__exit__()

. It also shows how Python calls them in a

with

construct:


>>> class HelloContextManager: ... def __enter__(self): ... print("Entering the context...") ... return "Hello, World!" ... def __exit__(self, exc_type, exc_value, exc_tb): ... print("Leaving the context...") ... print(exc_type, exc_value, exc_tb, sep="\n") ... >>> with HelloContextManager() as hello: ... print(hello) ... Entering the context... Hello, World! Leaving the context... None None None


HelloContextManager

implements both

.__enter__()

and

.__exit__()

. In

.__enter__()

, you first print a message to signal that the flow of execution is entering a new context. Then you return the

"Hello, World!"

string. In

.__exit__()

, you print a message to signal that the flow of execution is leaving the context. You also print the content of its three arguments.

When the

with

statement runs, Python creates a new instance of

HelloContextManager

and calls its

.__enter__()

method. You know this because you get

Entering the context...

printed on the screen.

Note: A common mistake when you’re using context managers is forgetting to call the object passed to the

with

statement.

In this case, the statement can’t get the required context manager, and you get an

AttributeError

like this:


>>> with HelloContextManager as hello: ... print(hello) ... Traceback (most recent call last): File "

", line 1, in

AttributeError: __enter__


The exception message doesn’t say too much, and you might feel confused in this kind of situation. So, make sure to call the object in the

with

statement to provide the corresponding context manager.

Then Python runs the

with

code block, which prints

hello

to the screen. Note that

hello

holds the return value of

.__enter__()

.

When the flow of execution exits the

with

code block, Python calls

.__exit__()

. You know that because you get

Leaving the context...

printed on your screen. The final line in the output confirms that the three arguments to

.__exit__()

are set to

None

.

Note: A common trick when you don’t remember the exact signature of

.__exit__()

and don’t need to access its arguments is to use

*args

and

**kwargs

like in

def __exit__(self, *args, **kwargs):

.

Now, what happens if an exception occurs during the execution of the

with

block? Go ahead and write the following

with

statement:


>>> with HelloContextManager() as hello: ... print(hello) ... hello[100] ... Entering the context... Hello, World! Leaving the context...

string index out of range
Traceback (most recent call last): File "

", line 3, in

IndexError: string index out of range



In this case, you try to retrieve the value at index

100

in the string

"Hello, World!"

. This raises an

IndexError

, and the arguments to

.__exit__()

are set to the following:


  • exc_type

    is the exception class,

    IndexError

    .

  • exc_value

    is the exception instance.

  • exc_tb

    is the traceback object.

This behavior is quite useful when you want to encapsulate the exception handling in your context managers.

Handling Exceptions in a Context Manager

As an example of encapsulating exception handling in a context manager, say you expect

IndexError

to be the most common exception when you’re working with

HelloContextManager

. You might want to handle that exception in the context manager so you don’t have to repeat the exception-handling code in every

with

code block. In that case, you can do something like this:


# exc_handling.py class HelloContextManager: def __enter__(self): print("Entering the context...") return "Hello, World!" def __exit__(self, exc_type, exc_value, exc_tb): print("Leaving the context...") if isinstance(exc_value, IndexError): # Handle IndexError here... print(f"An exception occurred in your with block: {exc_type}") print(f"Exception message: {exc_value}") return True with HelloContextManager() as hello: print(hello) hello[100] print("Continue normally from here...")

In

.__exit__()

, you check if

exc_value

is an instance of

IndexError

. If so, then you print a couple of informative messages and finally return with

True

. Returning a truthy value makes it possible to swallow the exception and continue the normal execution after the

with

code block.

In this example, if no

IndexError

occurs, then the method returns

None

and the exception propagates out. However, if you want to be more explicit, then you can return

False

from outside the

if

block.

If you run

exc_handling.py

from your command line, then you get the following output:


$ python exc_handling.py Entering the context... Hello, World! Leaving the context... An exception occurred in your with block:

Exception message: string index out of range Continue normally from here...


HelloContextManager

is now able to handle

IndexError

exceptions that occur in the

with

code block. Since you return

True

when an

IndexError

occurs, the flow of execution continues in the next line, right after exiting the

with

code block.

Opening Files for Writing: First Version

Now that you know how to implement the context management protocol, you can get a sense of what this would look like by coding a practical example. Here’s how you can take advantage of

open()

to create a context manager that opens files for writing:


# writable.py class WritableFile: def __init__(self, file_path): self.file_path = file_path def __enter__(self): self.file_obj = open(self.file_path, mode="w") return self.file_obj def __exit__(self, exc_type, exc_val, exc_tb): if self.file_obj: self.file_obj.close()


WritableFile

implements the context management protocol and supports the

with

statement, just like the original

open()

does, but it always opens the file for writing using the

"w"

mode. Here’s how you can use your new context manager:


>>> from writable import WritableFile >>> with WritableFile("hello.txt") as file: ... file.write("Hello, World!") ...

After running this code, your

hello.txt

file contains the

"Hello, World!"

string. As an exercise, you can write a complementary context manager that opens files for reading, but using

pathlib

functionalities. Go ahead and give it a shot!

Redirecting the Standard Output

A subtle detail to consider when you’re writing your own context managers is that sometimes you don’t have a useful object to return from

.__enter__()

and therefore to assign to the

with

target variable. In those cases, you can return

None

explicitly or you can just rely on Python’s implicit return value, which is

None

as well.

For example, say you need to temporarily redirect the standard output,

sys.stdout

, to a given file on your disk. To do this, you can create a context manager like this:


# redirect.py import sys class RedirectedStdout: def __init__(self, new_output): self.new_output = new_output def __enter__(self): self.saved_output = sys.stdout sys.stdout = self.new_output def __exit__(self, exc_type, exc_val, exc_tb): sys.stdout = self.saved_output

This context manager takes a file object through its constructor. In

.__enter__()

, you reassign the standard output,

sys.stdout

, to an instance attribute to avoid losing the reference to it. Then you reassign the standard output to point to the file on your disk. In

.__exit__()

, you just restore the standard output to its original value.

To use

RedirectedStdout

, you can do something like this:


>>> from redirect import RedirectedStdout >>> with open("hello.txt", "w") as file: ... with RedirectedStdout(file): ... print("Hello, World!") ... print("Back to the standard output...") ... Back to the standard output...

The outer

with

statement in this example provides the file object that you’re going to use as your new output,

hello.txt

. The inner

with

temporarily redirects the standard output to

hello.txt

, so the first call to

print()

writes directly to that file instead of printing

"Hello, World!"

on your screen. Note that when you leave the inner

with

code block, the standard output goes back to its original value.


RedirectedStdout

is a quick example of a context manager that doesn’t have a useful value to return from

.__enter__()

. However, if you’re only redirecting the

print()

output, you can get the same functionality without the need for coding a context manager. You just need to provide a

file

argument to

print()

like this:


>>> with open("hello.txt", "w") as file: ... print("Hello, World!", file=file) ...

In this examples,

print()

takes your

hello.txt

file as an argument. This causes

print()

to write directly into the physical file on your disk instead of printing

"Hello, World!"

to your screen.

Measuring Execution Time

Just like every other class, a context manager can encapsulate some internal state. The following example shows how to create a stateful context manager to measure the execution time of a given code block or function:


# timing.py from time import perf_counter class Timer: def __enter__(self): self.start = perf_counter() self.end = 0.0 return lambda: self.end - self.start def __exit__(self, *args): self.end = perf_counter()

When you use

Timer

in a

with

statement,

.__enter__()

gets called. This method uses

time.perf_counter()

to get the time at the beginning of the

with

code block and stores it in

.start

. It also initializes

.end

and returns a

lambda

function that computes a time delta. In this case,

.start

holds the initial state or time measurement.

Note: To take a deeper dive into how to time your code, check out Python Timer Functions: Three Ways to Monitor Your Code.

Once the

with

block ends,

.__exit__()

gets called. The method gets the time at the end of the block and updates the value of

.end

so that the

lambda

function can compute the time required to run the

with

code block.

Here’s how you can use this context manager in your code:


>>> from time import sleep >>> from timing import Timer >>> with Timer() as timer: ... # Time-consuming code goes here... ... sleep(0.5) ... >>> timer() 0.5005456680000862

With

Timer

, you can measure the execution time of any piece of code. In this example,

timer

holds an instance of the

lambda

function that computes the time delta, so you need to call

timer()

to get the final result.

i loved you , i might be able to talk about it more / romantic playlist
i loved you , i might be able to talk about it more / romantic playlist

Python3


class


FileManager():


def


__init__(


self


, filename, mode):


self


.filename


filename


self


.mode


mode


self


file


None


def


__enter__(


self


):


self


file


open


self


.filename,


self


.mode)


return


self


file


def


__exit__(


self


, exc_type, exc_value, exc_traceback):


self


file


.close()


with FileManager(


'test.txt'


'w'


) as f:


f.write(


'Test'


print


(f.closed)

Output:

True

File management using context manager and with statement: On executing the with block, the following operations happen in sequence:

  • A FileManager object is created with test.txt as the filename and w(write) as the mode when __init__ method is executed.
  • The __enter__ method opens the test.txt file in write mode(setup operation) and returns a file object to variable f.
  • The text ‘Test’ is written into the file.
  • The __exit__ method takes care of closing the file on exiting the with block(teardown operation). When print(f.closed) is run, the output is True as the FileManager has already taken care of closing the file which otherwise needed to be explicitly done.

Database connection management using context manager: Let’s create a simple database connection management system. The number of database connections that can be opened at a time is also limited(just like file descriptors). Therefore context managers are helpful in managing connections to the database as there could be chances that the programmer may forget to close the connection.

Conclusion

Context managers in Python is one of those topics that a lot of programmers have used but do not understand clearly.

I hope this article has cleared up some of your confusions.

If you’d like to connect to me, I am always available on LinkedIn. Feel free to shoot a message and I’d be happy to respond. Also, if you think this was helpful, consider endorsing my relevant skills on the platform.

Until the next one, take care and keep exploring.

contextlib — Utilities for with-statement contexts¶

Source code: Lib/contextlib.py

This module provides utilities for common tasks involving the

with

statement. For more information see also Context Manager Types and
With Statement Context Managers.

من دولار ل مليون!!.. كيف تحمي أموالك من التضخم وتستثمرها بنجاح؟
من دولار ل مليون!!.. كيف تحمي أموالك من التضخم وتستثمرها بنجاح؟

Implementing Python context manager protocol

The following shows a simple implementation of the

open()

function using the context manager protocol:


class File: def __init__(self, filename, mode): self.filename = filename self.mode = mode def __enter__(self): print(f'Opening the file {self.filename}.') self.__file = open(self.filename, self.mode) return self.__file def __exit__(self, exc_type, exc_value, exc_traceback): print(f'Closing the file {self.filename}.') if not self.__file.closed: self.__file.close() return False with File('data.txt', 'r') as f: print(int(next(f)))

Code language: Python (python)

How it works.

  • First, initialize the

    filename

    and

    mode

    in the

    __init__()

    method.
  • Second, open the file in the

    __enter__()

    method and return the file object.
  • Third, close the file if it’s open in the

    __exit__()

    method.

Utilities¶

Functions and classes provided:

class contextlib.AbstractContextManager¶

An abstract base class for classes that implement


object.__enter__()

and

object.__exit__()

. A default implementation for

object.__enter__()

is provided which returns

self

while

object.__exit__()

is an abstract method which by default returns

None

. See also the definition of Context Manager Types.

New in version 3.6.

class contextlib.AbstractAsyncContextManager¶

An abstract base class for classes that implement


object.__aenter__()

and

object.__aexit__()

. A default implementation for

object.__aenter__()

is provided which returns

self

while

object.__aexit__()

is an abstract method which by default returns

None

. See also the definition of Asynchronous Context Managers.

New in version 3.7.

@contextlib.contextmanager¶

This function is a decorator that can be used to define a factory function for


with

statement context managers, without needing to create a class or separate

__enter__()

and

__exit__()

methods.

While many objects natively support use in with statements, sometimes a resource needs to be managed that isn’t a context manager in its own right, and doesn’t implement a


close()

method for use with

contextlib.closing

An abstract example would be the following to ensure correct resource management:

from contextlib import contextmanager @contextmanager def managed_resource(*args, **kwds): # Code to acquire resource, e.g.: resource = acquire_resource(*args, **kwds) try: yield resource finally: # Code to release resource, e.g.: release_resource(resource)

The function can then be used like this:

>>> with managed_resource(timeout=3600) as resource: … # Resource is released at the end of this block, … # even if code in the block raises an exception

The function being decorated must return a generator-iterator when called. This iterator must yield exactly one value, which will be bound to the targets in the


with

statement’s

as

clause, if any.

At the point where the generator yields, the block nested in the


with

statement is executed. The generator is then resumed after the block is exited. If an unhandled exception occurs in the block, it is reraised inside the generator at the point where the yield occurred. Thus, you can use a

try



except



finally

statement to trap the error (if any), or ensure that some cleanup takes place. If an exception is trapped merely in order to log it or to perform some action (rather than to suppress it entirely), the generator must reraise that exception. Otherwise the generator context manager will indicate to the

with

statement that the exception has been handled, and execution will resume with the statement immediately following the

with

statement.

contextmanager()

uses

ContextDecorator

so the context managers it creates can be used as decorators as well as in

with

statements. When used as a decorator, a new generator instance is implicitly created on each function call (this allows the otherwise “one-shot” context managers created by

contextmanager()

to meet the requirement that context managers support multiple invocations in order to be used as decorators).

Changed in version 3.2: Use of


ContextDecorator

.

@contextlib.asynccontextmanager¶

Similar to


contextmanager()

, but creates an asynchronous context manager.

This function is a decorator that can be used to define a factory function for


async with

statement asynchronous context managers, without needing to create a class or separate

__aenter__()

and

__aexit__()

methods. It must be applied to an asynchronous generator function.

A simple example:

from contextlib import asynccontextmanager @asynccontextmanager async def get_connection(): conn = await acquire_db_connection() try: yield conn finally: await release_db_connection(conn) async def get_all_users(): async with get_connection() as conn: return conn.query(‘SELECT …’)

New in version 3.7.

Context managers defined with


asynccontextmanager()

can be used either as decorators or with

async with

statements:

import time from contextlib import asynccontextmanager @asynccontextmanager async def timeit(): now = time.monotonic() try: yield finally: print(f’it took {time.monotonic() – now}s to run’) @timeit() async def main(): # … async code …

When used as a decorator, a new generator instance is implicitly created on each function call. This allows the otherwise “one-shot” context managers created by


asynccontextmanager()

to meet the requirement that context managers support multiple invocations in order to be used as decorators.

Changed in version 3.10: Async context managers created with


asynccontextmanager()

can be used as decorators.

contextlib.closing(thing)¶

Return a context manager that closes thing upon completion of the block. This is basically equivalent to:

from contextlib import contextmanager @contextmanager def closing(thing): try: yield thing finally: thing.close()

And lets you write code like this:

from contextlib import closing from urllib.request import urlopen with closing(urlopen(‘https://www.python.org’)) as page: for line in page: print(line)

without needing to explicitly close


page

. Even if an error occurs,

page.close()

will be called when the

with

block is exited.

Note

Most types managing resources support the context manager protocol, which closes thing on leaving the


with

statement. As such,

closing()

is most useful for third party types that don’t support context managers. This example is purely for illustration purposes, as

urlopen()

would normally be used in a context manager.

contextlib.aclosing(thing)¶

Return an async context manager that calls the


aclose()

method of thing upon completion of the block. This is basically equivalent to:

from contextlib import asynccontextmanager @asynccontextmanager async def aclosing(thing): try: yield thing finally: await thing.aclose()

Significantly,


aclosing()

supports deterministic cleanup of async generators when they happen to exit early by

break

or an exception. For example:

from contextlib import aclosing async with aclosing(my_generator()) as values: async for value in values: if value == 42: break

This pattern ensures that the generator’s async exit code is executed in the same context as its iterations (so that exceptions and context variables work as expected, and the exit code isn’t run after the lifetime of some task it depends on).

New in version 3.10.

contextlib.nullcontext(enter_result=None)¶

Return a context manager that returns enter_result from


__enter__

, but otherwise does nothing. It is intended to be used as a stand-in for an optional context manager, for example:

def myfunction(arg, ignore_exceptions=False): if ignore_exceptions: # Use suppress to ignore all exceptions. cm = contextlib.suppress(Exception) else: # Do not ignore any exceptions, cm has no effect. cm = contextlib.nullcontext() with cm: # Do something

An example using enter_result:

def process_file(file_or_path): if isinstance(file_or_path, str): # If string, open file cm = open(file_or_path) else: # Caller is responsible for closing file cm = nullcontext(file_or_path) with cm as file: # Perform processing on the file

It can also be used as a stand-in for asynchronous context managers:

async def send_http(session=None): if not session: # If no http session, create it with aiohttp cm = aiohttp.ClientSession() else: # Caller is responsible for closing the session cm = nullcontext(session) async with cm as session: # Send http requests with session

New in version 3.7.

Changed in version 3.10: asynchronous context manager support was added.

contextlib.suppress(*exceptions)¶

Return a context manager that suppresses any of the specified exceptions if they occur in the body of a


with

statement and then resumes execution with the first statement following the end of the

with

statement.

As with any other mechanism that completely suppresses exceptions, this context manager should be used only to cover very specific errors where silently continuing with program execution is known to be the right thing to do.

For example:

from contextlib import suppress with suppress(FileNotFoundError): os.remove(‘somefile.tmp’) with suppress(FileNotFoundError): os.remove(‘someotherfile.tmp’)

This code is equivalent to:

try: os.remove(‘somefile.tmp’) except FileNotFoundError: pass try: os.remove(‘someotherfile.tmp’) except FileNotFoundError: pass

This context manager is reentrant.

If the code within the


with

block raises a

BaseExceptionGroup

, suppressed exceptions are removed from the group. If any exceptions in the group are not suppressed, a group containing them is re-raised.

New in version 3.4.

Changed in version 3.12:


suppress

now supports suppressing exceptions raised as part of an

BaseExceptionGroup

.

contextlib.redirect_stdout(new_target)¶

Context manager for temporarily redirecting


sys.stdout

to another file or file-like object.

This tool adds flexibility to existing functions or classes whose output is hardwired to stdout.

For example, the output of


help()

normally is sent to sys.stdout. You can capture that output in a string by redirecting the output to an

io.StringIO

object. The replacement stream is returned from the

__enter__

method and so is available as the target of the

with

statement:

with redirect_stdout(io.StringIO()) as f: help(pow) s = f.getvalue()

To send the output of


help()

to a file on disk, redirect the output to a regular file:

with open(‘help.txt’, ‘w’) as f: with redirect_stdout(f): help(pow)

To send the output of


help()

to sys.stderr:

with redirect_stdout(sys.stderr): help(pow)

Note that the global side effect on


sys.stdout

means that this context manager is not suitable for use in library code and most threaded applications. It also has no effect on the output of subprocesses. However, it is still a useful approach for many utility scripts.

This context manager is reentrant.

New in version 3.4.

contextlib.redirect_stderr(new_target)¶

Similar to


redirect_stdout()

but redirecting

sys.stderr

to another file or file-like object.

This context manager is reentrant.

New in version 3.5.

contextlib.chdir(path)¶

Non parallel-safe context manager to change the current working directory. As this changes a global state, the working directory, it is not suitable for use in most threaded or async contexts. It is also not suitable for most non-linear code execution, like generators, where the program execution is temporarily relinquished – unless explicitly desired, you should not yield when this context manager is active.

This is a simple wrapper around


chdir()

, it changes the current working directory upon entering and restores the old one on exit.

This context manager is reentrant.

New in version 3.11.

class contextlib.ContextDecorator¶

A base class that enables a context manager to also be used as a decorator.

Context managers inheriting from


ContextDecorator

have to implement

__enter__

and

__exit__

as normal.

__exit__

retains its optional exception handling even when used as a decorator.

ContextDecorator

is used by

contextmanager()

, so you get this functionality automatically.

Example of


ContextDecorator

:

from contextlib import ContextDecorator class mycontext(ContextDecorator): def __enter__(self): print(‘Starting’) return self def __exit__(self, *exc): print(‘Finishing’) return False

The class can then be used like this:

>>> @mycontext() … def function(): … print(‘The bit in the middle’) … >>> function() Starting The bit in the middle Finishing >>> with mycontext(): … print(‘The bit in the middle’) … Starting The bit in the middle Finishing

This change is just syntactic sugar for any construct of the following form:

def f(): with cm(): # Do stuff


ContextDecorator

lets you instead write:

@cm() def f(): # Do stuff

It makes it clear that the


cm

applies to the whole function, rather than just a piece of it (and saving an indentation level is nice, too).

Existing context managers that already have a base class can be extended by using


ContextDecorator

as a mixin class:

from contextlib import ContextDecorator class mycontext(ContextBaseClass, ContextDecorator): def __enter__(self): return self def __exit__(self, *exc): return False

Note

As the decorated function must be able to be called multiple times, the underlying context manager must support use in multiple


with

statements. If this is not the case, then the original construct with the explicit

with

statement inside the function should be used.

New in version 3.2.

class contextlib.AsyncContextDecorator¶

Similar to


ContextDecorator

but only for asynchronous functions.

Example of


AsyncContextDecorator

:

from asyncio import run from contextlib import AsyncContextDecorator class mycontext(AsyncContextDecorator): async def __aenter__(self): print(‘Starting’) return self async def __aexit__(self, *exc): print(‘Finishing’) return False

The class can then be used like this:

>>> @mycontext() … async def function(): … print(‘The bit in the middle’) … >>> run(function()) Starting The bit in the middle Finishing >>> async def function(): … async with mycontext(): … print(‘The bit in the middle’) … >>> run(function()) Starting The bit in the middle Finishing

New in version 3.10.

class contextlib.ExitStack¶

A context manager that is designed to make it easy to programmatically combine other context managers and cleanup functions, especially those that are optional or otherwise driven by input data.

For example, a set of files may easily be handled in a single with statement as follows:

with ExitStack() as stack: files = [stack.enter_context(open(fname)) for fname in filenames] # All opened files will automatically be closed at the end of # the with statement, even if attempts to open files later # in the list raise an exception

The


__enter__()

method returns the

ExitStack

instance, and performs no additional operations.

Each instance maintains a stack of registered callbacks that are called in reverse order when the instance is closed (either explicitly or implicitly at the end of a


with

statement). Note that callbacks are not invoked implicitly when the context stack instance is garbage collected.

This stack model is used so that context managers that acquire their resources in their


__init__

method (such as file objects) can be handled correctly.

Since registered callbacks are invoked in the reverse order of registration, this ends up behaving as if multiple nested


with

statements had been used with the registered set of callbacks. This even extends to exception handling – if an inner callback suppresses or replaces an exception, then outer callbacks will be passed arguments based on that updated state.

This is a relatively low level API that takes care of the details of correctly unwinding the stack of exit callbacks. It provides a suitable foundation for higher level context managers that manipulate the exit stack in application specific ways.

New in version 3.3.

enter_context(cm)¶

Enters a new context manager and adds its


__exit__()

method to the callback stack. The return value is the result of the context manager’s own

__enter__()

method.

These context managers may suppress exceptions just as they normally would if used directly as part of a


with

statement.

Changed in version 3.11: Raises


TypeError

instead of

AttributeError

if cm is not a context manager.

push(exit)¶

Adds a context manager’s


__exit__()

method to the callback stack.

As


__enter__

is not invoked, this method can be used to cover part of an

__enter__()

implementation with a context manager’s own

__exit__()

method.

If passed an object that is not a context manager, this method assumes it is a callback with the same signature as a context manager’s


__exit__()

method and adds it directly to the callback stack.

By returning true values, these callbacks can suppress exceptions the same way context manager


__exit__()

methods can.

The passed in object is returned from the function, allowing this method to be used as a function decorator.

callback(callback, /, *args, **kwds)¶

Accepts an arbitrary callback function and arguments and adds it to the callback stack.

Unlike the other methods, callbacks added this way cannot suppress exceptions (as they are never passed the exception details).

The passed in callback is returned from the function, allowing this method to be used as a function decorator.

pop_all()¶

Transfers the callback stack to a fresh


ExitStack

instance and returns it. No callbacks are invoked by this operation – instead, they will now be invoked when the new stack is closed (either explicitly or implicitly at the end of a

with

statement).

For example, a group of files can be opened as an “all or nothing” operation as follows:

with ExitStack() as stack: files = [stack.enter_context(open(fname)) for fname in filenames] # Hold onto the close method, but don’t call it yet. close_files = stack.pop_all().close # If opening any file fails, all previously opened files will be # closed automatically. If all files are opened successfully, # they will remain open even after the with statement ends. # close_files() can then be invoked explicitly to close them all.

close()¶

Immediately unwinds the callback stack, invoking callbacks in the reverse order of registration. For any context managers and exit callbacks registered, the arguments passed in will indicate that no exception occurred.

class contextlib.AsyncExitStack¶

An asynchronous context manager, similar to


ExitStack

, that supports combining both synchronous and asynchronous context managers, as well as having coroutines for cleanup logic.

The


close()

method is not implemented;

aclose()

must be used instead.

coroutine enter_async_context(cm)¶

Similar to


ExitStack.enter_context()

but expects an asynchronous context manager.

Changed in version 3.11: Raises


TypeError

instead of

AttributeError

if cm is not an asynchronous context manager.

push_async_exit(exit)¶

Similar to


ExitStack.push()

but expects either an asynchronous context manager or a coroutine function.

push_async_callback(callback, /, *args, **kwds)¶

Similar to


ExitStack.callback()

but expects a coroutine function.

coroutine aclose()¶

Similar to


ExitStack.close()

but properly handles awaitables.

Continuing the example for


asynccontextmanager()

:

async with AsyncExitStack() as stack: connections = [await stack.enter_async_context(get_connection()) for i in range(5)] # All opened connections will automatically be released at the end of # the async with statement, even if attempts to open a connection # later in the list raise an exception.

New in version 3.7.

Intermediate Python Programming Course
Intermediate Python Programming Course

Introduction to Python context managers

A context manager is an object that defines a runtime context executing within the

with

statement.

Let’s start with a simple example to understand the context manager concept.

Suppose that you have a file called

data.txt

that contains an integer

100

.

The following program reads the

data.txt

file, converts its contents to a number, and shows the result to the standard output:


f = open('data.txt') data = f.readlines() # convert the number to integer and display it print(int(data[0])) f.close()

Code language: Python (python)

The code is simple and straightforward.

However, the

data.txt

may contain data that cannot be converted to a number. In this case, the code will result in an exception.

For example, if the

data.txt

contains the string

'100'

instead of the number 100, you’ll get the following error:


ValueError: invalid literal for int() with base 10: "'100'"

Code language: Python (python)

Because of this exception, Python may not close the file properly.

To fix this, you may use the

try...except...finally

statement:


try: f = open('data.txt') data = f.readlines() # convert the number to integer and display it print(int(data[0])) except ValueError as error: print(error) finally: f.close()

Code language: Python (python)

Since the code in the

finally

block always executes, the code will always close the file properly.

This solution works as expected. However, it’s quite verbose.

Therefore, Python provides you with a better way that allows you to automatically close the file after you complete processing it.

This is where context managers come into play.

The following shows how to use a context manager to process the

data.txt

file:


with open('data.txt') as f: data = f.readlines() print(int(data[0])

Code language: Python (python)

In this example, we use the

open()

function with the

with

statement. After the

with

block, Python will close automatically.

Creating an Asynchronous Context Manager

To create an asynchronous context manager, you need to define the

.__aenter__()

and

.__aexit__()

methods. The script below is a reimplementation of the original script

site_checker_v0.py

you saw before, but this time you provide a custom asynchronous context manager to wrap the session creation and closing functionalities:


# site_checker_v1.py import aiohttp import asyncio class AsyncSession: def __init__(self, url): self._url = url async def __aenter__(self): self.session = aiohttp.ClientSession() response = await self.session.get(self._url) return response async def __aexit__(self, exc_type, exc_value, exc_tb): await self.session.close() async def check(url): async with AsyncSession(url) as response: print(f"{url}: status -> {response.status}") html = await response.text() print(f"{url}: type -> {html[:17].strip()}") async def main(): await asyncio.gather( check("https://realpython.com"), check("https://pycoders.com"), ) asyncio.run(main())

This script works similar to its previous version,

site_checker_v0.py

. The main difference is that, in this example, you extract the logic of the original outer

async with

statement and encapsulate it in

AsyncSession

.

In

.__aenter__()

, you create an

aiohttp.ClientSession()

, await the

.get()

response, and finally return the response itself. In

.__aexit__()

, you close the session, which corresponds to the teardown logic in this specific case. Note that

.__aenter__()

and

.__aexit__()

must return awaitable objects. In other words, you must define them with

async def

, which returns a coroutine object that is awaitable by definition.

If you run the script from your command line, then you get an output similar to this:


$ python site_checker_v1.py https://realpython.com: status -> 200 https://pycoders.com: status -> 200 https://realpython.com: type ->

https://pycoders.com: type ->

Great! Your script works just like its first version. It sends

GET

requests to both sites concurrently and processes the corresponding responses.

Finally, a common practice when you’re writing asynchronous context managers is to implement the four special methods:


  1. .__aenter__()

  2. .__aexit__()

  3. .__enter__()

  4. .__exit__()

This makes your context manager usable with both variations of

with

.

Python API Development - Comprehensive Course for Beginners
Python API Development – Comprehensive Course for Beginners

Python with statement

Here is the typical syntax of the

with

statement:


with context as ctx: # use the the object # context is cleaned up

Code language: Python (python)

How it works.

  • When Python encounters the

    with

    statement, it creates a new context. The context can optionally return an

    object

    .
  • After the

    with

    block, Python cleans up the context automatically.
  • The scope of the

    ctx

    has the same scope as the

    with

    statement. It means that you can access the

    ctx

    both inside and after the

    with

    statement.

The following shows how to access the variable after the

with

statement:


with open('data.txt') as f: data = f.readlines() print(int(data[0])) print(f.closed) # True

Code language: Python (python)

The with statement

A

with

statement is the primary method used to call a context manager. The general syntax is as follows:

This code works in the following order:


  1. SomeContextManager

    executes its enter (setup) logic before the indented code runs.

  2. SomeContextManager

    binds a value to

    context_variable

    , which can be used in the indented code
  3. Inner code block runs:

    # do stuff

    .

  4. SomeContextManager

    executes its exit (cleanup) logic.

So why is this useful? In the next section, we’ll discuss why this programming pattern comes in handy and why it’s worth making your context managers from time to time.

Build a Comment Toxicity Model with Deep Learning and Python
Build a Comment Toxicity Model with Deep Learning and Python

Examples and Recipes¶

This section describes some examples and recipes for making effective use of
the tools provided by

contextlib

.

Supporting a variable number of context managers¶

The primary use case for

ExitStack

is the one given in the class
documentation: supporting a variable number of context managers and other
cleanup operations in a single

with

statement. The variability
may come from the number of context managers needed being driven by user
input (such as opening a user specified collection of files), or from
some of the context managers being optional:

with ExitStack() as stack: for resource in resources: stack.enter_context(resource) if need_special_resource(): special = acquire_special_resource() stack.callback(release_special_resource, special) # Perform operations that use the acquired resources

As shown,

ExitStack

also makes it quite easy to use

with

statements to manage arbitrary resources that don’t natively support the
context management protocol.

Catching exceptions from __enter__ methods¶

It is occasionally desirable to catch exceptions from an

__enter__

method implementation, without inadvertently catching exceptions from
the

with

statement body or the context manager’s

__exit__

method. By using

ExitStack

the steps in the context management
protocol can be separated slightly in order to allow this:

stack = ExitStack() try: x = stack.enter_context(cm) except Exception: # handle __enter__ exception else: with stack: # Handle normal case

Actually needing to do this is likely to indicate that the underlying API
should be providing a direct resource management interface for use with

try

/

except

/

finally

statements, but not
all APIs are well designed in that regard. When a context manager is the
only resource management API provided, then

ExitStack

can make it
easier to handle various situations that can’t be handled directly in a

with

statement.

Cleaning up in an __enter__ implementation¶

As noted in the documentation of

ExitStack.push()

, this
method can be useful in cleaning up an already allocated resource if later
steps in the

__enter__()

implementation fail.

Here’s an example of doing this for a context manager that accepts resource acquisition and release functions, along with an optional validation function, and maps them to the context management protocol:

from contextlib import contextmanager, AbstractContextManager, ExitStack class ResourceManager(AbstractContextManager): def __init__(self, acquire_resource, release_resource, check_resource_ok=None): self.acquire_resource = acquire_resource self.release_resource = release_resource if check_resource_ok is None: def check_resource_ok(resource): return True self.check_resource_ok = check_resource_ok @contextmanager def _cleanup_on_error(self): with ExitStack() as stack: stack.push(self) yield # The validation check passed and didn’t raise an exception # Accordingly, we want to keep the resource, and pass it # back to our caller stack.pop_all() def __enter__(self): resource = self.acquire_resource() with self._cleanup_on_error(): if not self.check_resource_ok(resource): msg = “Failed validation for {!r}” raise RuntimeError(msg.format(resource)) return resource def __exit__(self, *exc_details): # We don’t need to duplicate any of our resource release logic self.release_resource()

Replacing any use of try-finally and flag variables¶

A pattern you will sometimes see is a

try-finally

statement with a flag
variable to indicate whether or not the body of the

finally

clause should
be executed. In its simplest form (that can’t already be handled just by
using an

except

clause instead), it looks something like this:

cleanup_needed = True try: result = perform_operation() if result: cleanup_needed = False finally: if cleanup_needed: cleanup_resources()

As with any

try

statement based code, this can cause problems for
development and review, because the setup code and the cleanup code can end
up being separated by arbitrarily long sections of code.


ExitStack

makes it possible to instead register a callback for
execution at the end of a

with

statement, and then later decide to skip
executing that callback:

from contextlib import ExitStack with ExitStack() as stack: stack.callback(cleanup_resources) result = perform_operation() if result: stack.pop_all()

This allows the intended cleanup up behaviour to be made explicit up front, rather than requiring a separate flag variable.

If a particular application uses this pattern a lot, it can be simplified even further by means of a small helper class:

from contextlib import ExitStack class Callback(ExitStack): def __init__(self, callback, /, *args, **kwds): super().__init__() self.callback(callback, *args, **kwds) def cancel(self): self.pop_all() with Callback(cleanup_resources) as cb: result = perform_operation() if result: cb.cancel()

If the resource cleanup isn’t already neatly bundled into a standalone
function, then it is still possible to use the decorator form of

ExitStack.callback()

to declare the resource cleanup in
advance:

from contextlib import ExitStack with ExitStack() as stack: @stack.callback def cleanup_resources(): … result = perform_operation() if result: stack.pop_all()

Due to the way the decorator protocol works, a callback function declared this way cannot take any parameters. Instead, any resources to be released must be accessed as closure variables.

Using a context manager as a function decorator¶


ContextDecorator

makes it possible to use a context manager in
both an ordinary

with

statement and also as a function decorator.

For example, it is sometimes useful to wrap functions or groups of statements
with a logger that can track the time of entry and time of exit. Rather than
writing both a function decorator and a context manager for the task,
inheriting from

ContextDecorator

provides both capabilities in a
single definition:

from contextlib import ContextDecorator import logging logging.basicConfig(level=logging.INFO) class track_entry_and_exit(ContextDecorator): def __init__(self, name): self.name = name def __enter__(self): logging.info(‘Entering: %s’, self.name) def __exit__(self, exc_type, exc, exc_tb): logging.info(‘Exiting: %s’, self.name)

Instances of this class can be used as both a context manager:

with track_entry_and_exit(‘widget loader’): print(‘Some time consuming activity goes here’) load_widget()

And also as a function decorator:

@track_entry_and_exit(‘widget loader’) def activity(): print(‘Some time consuming activity goes here’) load_widget()

Note that there is one additional limitation when using context managers
as function decorators: there’s no way to access the return value of

__enter__()

. If that value is needed, then it is still necessary to use
an explicit

with

statement.

Creating Context Managers

Function-Based Implementation

The standard library provides

contextlib

, a module containing

with

statement utilities. One important helper is the

@contextmanager

decorator, which lets us define function-based context managers.

Let’s try it out by making a context manager function that swaps the case of a string:

The

@contextmanager

decorator expects a

yield

statement in the function body.

yield

divides the function into three parts:

  1. The expressions above

    yield

    are executed right before the code that is managed.
  2. The managed code runs at

    yield

    . Whatever is yielded makes up the context variable—

    swapped_string

    in our case.
  3. The expressions below

    yield

    are executed after the managed code is run.

Notice that we have placed

yield

within a

try-except-finally

block. This is not enforced, yet, it is good practice. This way, if an error occurs in the managed code the context manager will carry out its exit logic no matter what.

Look what happens if we raise a

ValueError

inside the

with

statement:

The

except

block within

example_cm

handled the exception, and the

finally

block ensured we saw the exit message.

Notice that the “End of context manager” string is still printed. We’re specifically catching a

ValueError

, but let’s see what happens when an unhandled error is raised:

In this case, the

finally

block still executed regardless of the uncaught error, but the “End of context manager” printout didn’t make it. This exemplifies why a

finally

block is helpful in many situations.

For more flexibility in creating context managers, we’ll now introduce the class-based implementations.

Class-based implementation

Python defines a context management protocol that dictates that any class with an

__enter__()

and an

__exit__()

method can work as a context manager.

Double underscore methods are special methods that are not called but instead triggered. They are internally set to run at specific times or after certain events. For example,

__enter()__

runs when a

with

statement is entered and

__exit__

runs right before the

with

block is left.

We have seen an example of function-based context managers, which work great for quick and simple cases. For more complex use cases, we’ll define context management as an additional ability to an existing class.

Let’s create a

ListProtect

context manager class, which will operate on a copy of a list before returning the changes. This way, the original list would be restored unaltered if an error occurred during the operation.

Let’s see how that looks.

And we can use it like so:

The list was successfully modified. But what if an error occurs?

In this instance,

ListProtect

caught an error and the list remained unaltered. So how does this work?

Explanation

In the

ListProtect

class, we defined two required methods:


__enter__()

– this method defines what happens before the logic under the

with

statement runs. If the enter method returns anything, it is bound to the context variable. In our class, we create a clone of the list and returned it as the context variable—

the_copy

in the above examples.


__exit__()

– this method defines what happens when the

with

logic is complete or has raised an error. Besides

self

, this method takes three parameters:

exc_type

,

exc_val

,

exc_tb

, which can also be shortened to

*exc

. If no exception occurs in the managee, all these values are None. If we return a truthy value at the end of

__exit()__

—as we did in our class—the error is suppressed. Otherwise, if we return a falsy value, such as

None

,

False

, or blank, the error is propagated.

In short,

ListProtect

‘s

__exit()__

method first checks whether the exception type was

None

. If so, there were no errors, and it applies the changes to the original list; otherwise, it announces an error occurred.

Senior Programmers vs Junior Developers #shorts
Senior Programmers vs Junior Developers #shorts

Python3


from


pymongo


import


MongoClient


class


MongoDBConnectionManager():


def


__init__(


self


, hostname, port):


self


.hostname


hostname


self


.port


port


self


.connection


None


def


__enter__(


self


):


self


.connection


MongoClient(


self


.hostname,


self


.port)


return


self


.connection


def


__exit__(


self


, exc_type, exc_value, exc_traceback):


self


.connection.close()


with MongoDBConnectionManager(


'localhost'


'27017'


) as mongo:


collection


mongo.connection.SampleDb.test


data


collection.find({


'_id'


})


print


(data.get(


'name'


))

Database connection management using context manager and with statement: On executing the with block, the following operations happen in sequence:

  • A MongoDBConnectionManager object is created with localhost as the hostname name and 27017 as the port when the __init__ method is executed.
  • The __enter__ method opens the MongoDB connection and returns the MongoClient object to variable mongo.
  • The test collection in the SampleDb database is accessed and the document with _id=1 is retrieved. The name field of the document is printed.
  • The __exit__ method takes care of closing the connection on exiting the with block(teardown operation).

Don’t miss your chance to ride the wave of the data revolution! Every industry is scaling new heights by tapping into the power of data. Sharpen your skills and become a part of the hottest trend in the 21st century.

Dive into the future of technology – explore the Complete Machine Learning and Data Science Program by GeeksforGeeks and stay ahead of the curve.

Last Updated :
03 Nov, 2022

Like Article

Save Article

Share your thoughts in the comments

Please Login to comment…

One of the most common tasks that you’ll have to perform in your programs is working with external resources. These resources can be files on your computer’s storage or an open connection to third-party service on the internet.

For the sake of simplicity, imagine a program that opens a file, writes something to it, and then closes the file.

One way to implement this program in Python would be like this:


def main(): my_file = open('books.txt', 'w') my_file.write('If Tomorrow Comes by Sidney Sheldon') my_file.close() if __name__ == '__main__': main()

Given that you run this program with the right permissions on your computer, it’ll create a file called

books.txt

and write

If Tomorrow Comes by Sidney Sheldon

in it.

The

open()

function is one of the built-in functions in Python. It can open a file from a given path and return a corresponding file object.

A file object or file-like object, as it’s often called, is a useful way to encapsulate methods like

read()

,

write()

, or

close()

.

The

write()

method can be used write/send bytes-like object to an open stream, like a file.

Whenever you open an external resource, you must close it when its no longer needed, and the

close()

method does just that.

This program is functional, but it has a big flaw. If the program fails to close the file, it will remain open until the program itself closes.

You see, every program that you run on your computer gets a finite amount of memory allocated to it. All the variables you create or external resource you open from a program stay within the memory allocated to it by your computer.

If a program like this one, keeps opening new files without closing the previous ones, the allocated memory will keep shrinking.

At one point the program will inevitably run out of memory and crash ungracefully. This problem is referred to as a memory leak.

One way to prevent this from happening in Python is using a

try...except...finally

statement.


def main(): my_file = open('books.txt', 'w') try: my_file.write('If Tomorrow Comes by Sidney Sheldon') except Exception as e: print(f'writing to file failed: {e}') finally: my_file.close() if __name__ == '__main__': main()

The code inside the

finally

block will run no matter what. So even if the program fails on the right action, it’ll still be executed.

So, this solves the problem but imagine writing these lines of code every time you want to write something to a file.

It’s not very reusable. You will have to repeat yourself a lot and chances of skipping a portion of the

if...except...finally

ladder is also a possibility.

That’s where context managers come in.

Writing Good APIs With Context Managers

Context managers are quite flexible, and if you use the

with

statement creatively, then you can define convenient APIs for your classes, modules, and packages.

For example, what if the resource you wanted to manage is the text indentation level in some kind of report generator application? In that case, you could write code like this:


with Indenter() as indent: indent.print("hi!") with indent: indent.print("hello") with indent: indent.print("bonjour") indent.print("hey")

This almost reads like a domain-specific language (DSL) for indenting text. Also, notice how this code enters and leaves the same context manager multiple times to switch between different indentation levels. Running this code snippet leads to the following output and prints neatly formatted text:


hi! hello bonjour hey

How would you implement a context manager to support this functionality? This could be a great exercise to wrap your head around how context managers work. So, before you check out the implementation below, you might take some time and try to solve this by yourself as a learning exercise.

Ready? Here’s how you might implement this functionality using a context manager class:


class Indenter: def __init__(self): self.level = -1 def __enter__(self): self.level += 1 return self def __exit__(self, exc_type, exc_value, exc_tb): self.level -= 1 def print(self, text): print(" " * self.level + text)

Here,

.__enter__()

increments

.level

by every time the flow of execution enters the context. The method also returns the current instance,

self

. In

.__exit__()

, you decrease

.level

so the printed text steps back one indentation level every time you exit the context.

The key point in this example is that returning

self

from

.__enter__()

allows you to reuse the same context manager across several nested

with

statements. This changes the text indentation level every time you enter and leave a given context.

A good exercise for you at this point would be to write a function-based version of this context manager. Go ahead and give it a try!

Why Is Lua SO Popular?
Why Is Lua SO Popular?

Python context manager protocol

Python context managers work based on the context manager protocol.

The context manager protocol has the following methods:


  • __enter__()

    – setup the context and optionally return some object

  • __exit__()

    – cleanup the object.

If you want a class to support the context manager protocol, you need to implement these two methods.

Suppose that

ContextManager

is a class that supports the context manager protocol.

The following shows how to use the

ContextManager

class:


with ContextManager() as ctx: # do something # done with the context

Code language: Python (python)

When you use

ContextManager

class with the

with

statement, Python implicitly creates an instance of the

ContextManager

class (

instance

) and automatically call

__enter__()

method on that instance.

The

__enter__()

method may optionally return an object. If so, Python assigns the returned object the

ctx

.

Notice that

ctx

references the object returned by the

__enter__()

method. It doesn’t reference the instance of the

ContextManager

class.

If an exception occurs inside the with block or after the

with

block, Python calls the

__exit__()

method on the

instance

object.

Functionally, the

with

statement is equivalent to the following

try...finally

statement:


instance = ContextManager() ctx = instance.__enter__() try: # do something with the txt finally: # done with the context instance.__exit__()

Code language: Python (python)

The __enter__() method

In the

__enter__()

method, you can carry the necessary steps to setup the context.

Optionally, you can returns an object from the

__enter__()

method.

The __exit__() method

Python always executes the

__exit__()

method even if an exception occurs in the

with

block.

The

__exit__()

method accepts three arguments: exception type, exception value, and traceback object. All of these arguments will be

None

if no exception occurs.


def __exit__(self, ex_type, ex_value, ex_traceback): ...

Code language: Python (python)

The

__exit__()

method returns a boolean value, either

True

or

False

.

If the return value is True, Python will make any exception silent. Otherwise, it doesn’t silence the exception.

Python3


file_descriptors


[]


for


in


range


100000


):


file_descriptors.append(


open


'test.txt'


'w'


))

Output:

Traceback (most recent call last):
File “context.py”, line 3, in
OSError: [Errno 24] Too many open files: ‘test.txt’

An error message saying that too many files are open. The above example is a case of file descriptor leakage. It happens because there are too many open files and they are not closed. There might be chances where a programmer may forget to close an opened file.

Managing Resources using context manager: Suppose a block of code raises an exception or if it has a complex algorithm with multiple return paths, it becomes cumbersome to close a file in all the places. Generally in other languages when working with files try-except-finally is used to ensure that the file resource is closed after usage even if there is an exception. Python provides an easy way to manage resources: Context Managers. The with keyword is used. When it gets evaluated it should result in an object that performs context management. Context managers can be written using classes or functions(with decorators).

Creating a Context Manager: When creating context managers using classes, user need to ensure that the class has the methods: __enter__() and __exit__(). The __enter__() returns the resource that needs to be managed and the __exit__() does not return anything but performs the cleanup operations. First, let us create a simple class called ContextManager to understand the basic structure of creating context managers using classes, as shown below:

Context API là gì? Cách sử dụng useContext() trong React
Context API là gì? Cách sử dụng useContext() trong React

Creating Function-Based Context Managers

Python’s generator functions and the

contextlib.contextmanager

decorator provide an alternative and convenient way to implement the context management protocol. If you decorate an appropriately coded generator function with

@contextmanager

, then you get a function-based context manager that automatically provides both required methods,

.__enter__()

and

.__exit__()

. This can make your life more pleasant by saving you some boilerplate code.

The general pattern to create a context manager using

@contextmanager

along with a generator function goes like this:


>>> from contextlib import contextmanager >>> @contextmanager ... def hello_context_manager(): ... print("Entering the context...") ... yield "Hello, World!" ... print("Leaving the context...") ... >>> with hello_context_manager() as hello: ... print(hello) ... Entering the context... Hello, World! Leaving the context...

In this example, you can identify two visible sections in

hello_context_manager()

. Before the

yield

statement, you have the setup section. There, you can place the code that acquires the managed resources. Everything before the

yield

runs when the flow of execution enters the context.

After the

yield

statement, you have the teardown section, in which you can release the resources and do the cleanup. The code after

yield

runs at the end of the

with

block. The

yield

statement itself provides the object that will be assigned to the

with

target variable.

This implementation and the one that uses the context management protocol are practically equivalent. Depending on which one you find more readable, you might prefer one over the other. A downside of the function-based implementation is that it requires an understanding of advanced Python topics, such as decorators and generators.

The

@contextmanager

decorator reduces the boilerplate required to create a context manager. Instead of writing a whole class with

.__enter__()

and

.__exit__()

methods, you just need to implement a generator function with a single

yield

that produces whatever you want

.__enter__()

to return.

Opening Files for Writing: Second Version

You can use the

@contextmanager

to reimplement your

WritableFile

context manager. Here’s what rewriting it with this technique looks like:


>>> from contextlib import contextmanager >>> @contextmanager ... def writable_file(file_path): ... file = open(file_path, mode="w") ... try: ... yield file ... finally: ... file.close() ... >>> with writable_file("hello.txt") as file: ... file.write("Hello, World!") ...

In this case,

writable_file()

is a generator function that opens

file

for writing. Then it temporarily suspends its own execution and yields the resource so

with

can bind it to its target variable. When the flow of execution leaves the

with

code block, the function continues to execute and closes

file

correctly.

Mocking the Time

As a final example of how to create custom context managers with

@contextmanager

, say you’re testing a piece of code that works with time measurements. The code uses

time.time()

to get the current time measurement and do some further computations. Since time measurements vary, you decide to mock

time.time()

so you can test your code.

Here’s a function-based context manager that can help you do that:


>>> from contextlib import contextmanager >>> from time import time >>> @contextmanager ... def mock_time(): ... global time ... saved_time = time ... time = lambda: 42 ... yield ... time = saved_time ... >>> with mock_time(): ... print(f"Mocked time: {time()}") ... Mocked time: 42 >>> # Back to normal time >>> time() 1616075222.4410584

Inside

mock_time()

, you use a

global

statement to signal that you’re going to modify the global name

time

. Then you save the original

time()

function object in

saved_time

so you can safely restore it later. The next step is to monkey patch

time()

using a

lambda

function that always returns the same value,

42

.

The bare

yield

statement specifies that this context manager doesn’t have a useful object to send back to the

with

target variable for later use. After

yield

, you reset the global

time

to its original content.

When the execution enters the

with

block, any calls to

time()

return

42

. Once you leave the

with

code block, calls to

time()

return the expected current time. That’s it! Now you can test your time-related code.

Using Python context manager to implement the start and stop pattern

The following defines a

Timer

class that supports the context manager protocol:


from time import perf_counter class Timer: def __init__(self): self.elapsed = 0 def __enter__(self): self.start = perf_counter() return self def __exit__(self, exc_type, exc_value, exc_traceback): self.stop = perf_counter() self.elapsed = self.stop - self.start return False

Code language: Python (python)

How it works.

  • First, import the

    perf_counter

    from the

    time

    module.
  • Second, start the timer in the

    __enter__()

    method
  • Third, stop the timer in the

    __exit__()

    method and return the elapsed time.

Now, you can use the

Timer

class to measure the time needed to calculate the Fibonacci of 1000, one million times:


def fibonacci(n): f1 = 1 f2 = 1 for i in range(n-1): f1, f2 = f2, f1 + f2 return f1 with Timer() as timer: for _ in range(1, 1000000): fibonacci(1000) print(timer.elapsed)

Code language: Python (python)

If You’re Not Using Python DATA CLASSES Yet, You Should 🚀
If You’re Not Using Python DATA CLASSES Yet, You Should 🚀

Creating Custom Context Managers

You’ve already worked with context managers from the standard library and third-party libraries. There’s nothing special or magical about

open()

,

threading.Lock

,

decimal.localcontext()

, or the others. They just return objects that implement the context management protocol.

You can provide the same functionality by implementing both the

.__enter__()

and the

.__exit__()

special methods in your class-based context managers. You can also create custom function-based context managers using the

contextlib.contextmanager

decorator from the standard library and an appropriately coded generator function.

In general, context managers and the

with

statement aren’t limited to resource management. They allow you to provide and reuse common setup and teardown code. In other words, with context managers, you can perform any pair of operations that needs to be done before and after another operation or procedure, such as:

  • Open and close
  • Lock and release
  • Change and reset
  • Create and delete
  • Enter and exit
  • Start and stop
  • Setup and teardown

You can provide code to safely manage any of these pairs of operations in a context manager. Then you can reuse that context manager in

with

statements throughout your code. This prevents errors and reduces repetitive boilerplate code. It also makes your APIs safer, cleaner, and more user-friendly.

In the next two sections, you’ll learn the basics of creating class-based and function-based context managers.

Using the Python with Statement

As long as Python developers have incorporated the

with

statement into their coding practice, the tool has been shown to have several valuable use cases. More and more objects in the Python standard library now provide support for the context management protocol so you can use them in a

with

statement.

In this section, you’ll code some examples that show how to use the

with

statement with several classes both in the standard library and in third-party libraries.

Working With Files

So far, you’ve used

open()

to provide a context manager and manipulate files in a

with

construct. Opening files using the

with

statement is generally recommended because it ensures that open file descriptors are automatically closed after the flow of execution leaves the

with

code block.

As you saw before, the most common way to open a file using

with

is through the built-in

open()

:


with open("hello.txt", mode="w") as file: file.write("Hello, World!")

In this case, since the context manager closes the file after leaving the

with

code block, a common mistake might be the following:


>>> file = open("hello.txt", mode="w") >>> with file: ... file.write("Hello, World!") ... 13 >>> with file: ... file.write("Welcome to Real Python!") ... Traceback (most recent call last): File "

", line 1, in

ValueError: I/O operation on closed file.


The first

with

successfully writes

"Hello, World!"

into

hello.txt

. Note that

.write()

returns the number of bytes written into the file,

13

. When you try to run a second

with

, however, you get a

ValueError

because your

file

is already closed.

Another way to use the

with

statement to open and manage files is by using

pathlib.Path.open()

:


>>> import pathlib >>> file_path = pathlib.Path("hello.txt") >>> with file_path.open("w") as file: ... file.write("Hello, World!") ... 13


Path

is a class that represents concrete paths to physical files in your computer. Calling

.open()

on a

Path

object that points to a physical file opens it just like

open()

would do. So,

Path.open()

works similarly to

open()

, but the file path is automatically provided by the

Path

object you call the method on.

Since

pathlib

provides an elegant, straightforward, and Pythonic way to manipulate file system paths, you should consider using

Path.open()

in your

with

statements as a best practice in Python.

Finally, whenever you load an external file, your program should check for possible issues, such as a missing file, writing and reading access, and so on. Here’s a general pattern that you should consider using when you’re working with files:


import pathlib import logging file_path = pathlib.Path("hello.txt") try: with file_path.open(mode="w") as file: file.write("Hello, World!") except OSError as error: logging.error("Writing to file %s failed due to: %s", file_path, error)

In this example, you wrap the

with

statement in a

try



except

statement. If an

OSError

occurs during the execution of

with

, then you use

logging

to log the error with a user-friendly and descriptive message.

Traversing Directories

The

os

module provides a function called

scandir()

, which returns an iterator over

os.DirEntry

objects corresponding to the entries in a given directory. This function is specially designed to provide optimal performance when you’re traversing a directory structure.

A call to

scandir()

with the path to a given directory as an argument returns an iterator that supports the context management protocol:


>>> import os >>> with os.scandir(".") as entries: ... for entry in entries: ... print(entry.name, "->", entry.stat().st_size, "bytes") ... Documents -> 4096 bytes Videos -> 12288 bytes Desktop -> 4096 bytes DevSpace -> 4096 bytes .profile -> 807 bytes Templates -> 4096 bytes Pictures -> 12288 bytes Public -> 4096 bytes Downloads -> 4096 bytes

In this example, you write a

with

statement with

os.scandir()

as the context manager supplier. Then you iterate over the entries in the selected directory (

"."

) and print their name and size on the screen. In this case,

.__exit__()

calls

scandir.close()

to close the iterator and release the acquired resources. Note that if you run this on your machine, you’ll get a different output depending on the content of your current directory.

Performing High-Precision Calculations

Unlike built-in floating-point numbers, the

decimal

module provides a way to adjust the precision to use in a given calculation that involves

Decimal

numbers. The precision defaults to

28

places, but you can change it to meet your problem requirements. A quick way to perform calculations with a custom precision is using

localcontext()

from

decimal

:


>>> from decimal import Decimal, localcontext >>> with localcontext() as ctx: ... ctx.prec = 42 ... Decimal("1") / Decimal("42") ... Decimal('0.0238095238095238095238095238095238095238095') >>> Decimal("1") / Decimal("42") Decimal('0.02380952380952380952380952381')

Here,

localcontext()

provides a context manager that creates a local decimal context and allows you to perform calculations using a custom precision. In the

with

code block, you need to set

.prec

to the new precision you want to use, which is

42

places in the example above. When the

with

code block finishes, the precision is reset back to its default value,

28

places.

Handling Locks in Multithreaded Programs

Another good example of using the

with

statement effectively in the Python standard library is

threading.Lock

. This class provides a primitive lock to prevent multiple threads from modifying a shared resource at the same time in a multithreaded application.

You can use a

Lock

object as the context manager in a

with

statement to automatically acquire and release a given lock. For example, say you need to protect the balance of a bank account:


import threading balance_lock = threading.Lock() # Use the try ... finally pattern balance_lock.acquire() try: # Update the account balance here ... finally: balance_lock.release() # Use the with pattern with balance_lock: # Update the account balance here ...

The

with

statement in the second example automatically acquires and releases a lock when the flow of execution enters and leaves the statement. This way, you can focus on what really matters in your code and forget about those repetitive operations.

In this example, the lock in the

with

statement creates a protected region known as the critical section, which prevents concurrent access to the account balance.

Testing for Exceptions With pytest

So far, you’ve coded a few examples using context managers that are available in the Python standard library. However, several third-party libraries include objects that support the context management protocol.

Say you’re testing your code with pytest. Some of your functions and code blocks raise exceptions under certain situations, and you want to test those cases. To do that, you can use

pytest.raises()

. This function allows you to assert that a code block or a function call raises a given exception.

Since

pytest.raises()

provides a context manager, you can use it in a

with

statement like this:


>>> import pytest >>> 1 / 0 Traceback (most recent call last): File "

", line 1, in

ZeroDivisionError: division by zero >>> with pytest.raises(ZeroDivisionError): ... 1 / 0 ... >>> favorites = {"fruit": "apple", "pet": "dog"} >>> favorites["car"] Traceback (most recent call last): File "

", line 1, in

KeyError: 'car' >>> with pytest.raises(KeyError): ... favorites["car"] ...




In the first example, you use

pytest.raises()

to capture the

ZeroDivisionError

that the expression

1 / 0

raises. The second example uses the function to capture the

KeyError

that is raised when you access a key that doesn’t exist in a given dictionary.

If your function or code block doesn’t raise the expected exception, then

pytest.raises()

raises a failure exception:


>>> import pytest >>> with pytest.raises(ZeroDivisionError): ... 4 / 2 ... 2.0 Traceback (most recent call last): ... Failed: DID NOT RAISE

Another cool feature of

pytest.raises()

is that you can specify a target variable to inspect the raised exception. For example, if you want to verify the error message, then you can do something like this:


>>> with pytest.raises(ZeroDivisionError) as exc: ... 1 / 0 ... >>> assert str(exc.value) == "division by zero"

You can use all these

pytest.raises()

features to capture the exceptions you raise from your functions and code block. This is a cool and useful tool that you can incorporate into your current testing strategy.

Python Full Course for free 🐍
Python Full Course for free 🐍

How To Create a Custom Context Manager in Python

The American theoretical physicist, Richard Feynman famously said —

What I cannot create, I do not understand.

So, to understand the functionalities of a context manager you must create one by yourself and there are two distinct ways of doing that.

The first one is a generator-based approach and the second one is a class-based approach. In this section, I’ll teach you both.

But before that, let me you a complex example that does more than merely opening and closing files in Python.

Imagine another Python application that must communicate with an SQLite database for reading and writing data.

You can write that program as follows:


import sqlite3 create_table_sql_statement = 'CREATE TABLE IF NOT EXISTS books(title TEXT, author TEXT)' insert_into_table_sql_statement = "INSERT INTO books VALUES ('If Tomorrow Comes', 'Sidney Sheldon'), ('The Lincoln Lawyer', 'Michael Connelly')" select_from_table_sql_statement = 'SELECT * FROM books' def main(): database_path = ':memory:' connection = sqlite3.connect(database_path) cursor = connection.cursor() try: cursor.execute(create_table_sql_statement) connection.commit() cursor.execute(insert_into_table_sql_statement) connection.commit() cursor.execute(select_from_table_sql_statement) print(cursor.fetchall()) except Exception as e: print(f'read or write action to the database failed: {e}') finally: connection.close() if __name__ == '__main__': main() # [('If Tomorrow Comes', 'Sidney Sheldon'), ('The Lincoln Lawyer', 'Michael Connelly')]

This Python program establishes a connection with an SQLite database. Then it creates a new table called books with two

TEXT

columns named

title

and

author

.

The program then stores information about three books on the table, retrieves them from the database, and prints out the retrieved data on the console.

As evident from the output of the

print()

statement, the program has successfully saved and retrieved the given data from the database.

There are three SQL queries in this program responsible for the database actions I just described.


create_table_sql_statement = 'CREATE TABLE IF NOT EXISTS books(title TEXT, author TEXT)' insert_into_table_sql_statement = "INSERT INTO books VALUES ('If Tomorrow Comes', 'Sidney Sheldon'), ('The Lincoln Lawyer', 'Michael Connelly')" select_from_table_sql_statement = 'SELECT * FROM books'

I’ve kept these three lines of code at the top of the file to keep the

main()

function to cleaner. The rest of the program sets up the database and executes the queries.

Python comes with excellent support for SQLite databases, thanks to the

sqlite3

module encapsulating useful methods such as the

sqlite3.connect()

method.

This method takes the path to a database as a string, attempts to establish a connection and in case of success, returns a

Connection

object.

If you pass

:memory:

instead of a file path, the program will create a temporary database on your computer’s memory.

Once you have a connection, you’ll need a

Cursor

object. A cursor object is a layer of abstraction required for executing SQL queries.

The

cursor()

method encapsulated within the

Connection

object returns a new cursor to the connected database.

Inside a

try

block, you can attempt to execute whatever query you want using the

execute()

or

executemany()

methods.


try: cursor.execute(create_table_sql_statement) connection.commit() cursor.execute(insert_into_table_sql_statement) connection.commit() cursor.execute(select_from_table_sql_statement) print(cursor.fetchall())

You need to call the

connection.commit()

method every time you write something to the database. Otherwise, the changes will be lost.

Data returned from a database remains within the

cursor

object and you can access them using the

cursor.fetchone()

or

cursor.fetchall()

methods.

In case of a failure, the

except

block will be triggered. The

finally

block will run unconditionally and close the database connection in the end.

This is fine and functional but like I’ve already said, it’s not very reusable and is error prone.

Unfortunately, or in our case fortunately Python doesn’t come with a built-in context manager for handling connections with SQLite databases.

So, let’s try and see if we can produce one ourselves.

How to Create a Class Based Context Manager in Python

To write a class-based context manager in Python, you need to create an empty class with three specific methods:


class Database: def __init__(self): pass def __enter__(self): pass def __exit__(self, exc_type, exc_val, exc_tb): pass

The first one is obviously the class constructor that doesn’t accept any parameter yet. It’ll be responsible for accepting a database path:


import sqlite3 class Database: def __init__(self, path: str): self.path = path def __enter__(self): pass def __exit__(self, exc_type, exc_val, exc_tb): pass

The

__enter__()

method handles the task of setting up the resource. This is where you establish the connection and instantiate the cursor:


import sqlite3 class Database: def __init__(self, path: str): self.path = path def __enter__(self): self.connection = sqlite3.connect(self.path) self.cursor = self.connection.cursor() return self def __exit__(self, exc_type, exc_val, exc_tb): pass

However you can not return two objects at once so you have to return the instance of the class itself.

Finally, the

__exit__()

method handles the task of closing the external resource in question.


import sqlite3 class Database: def __init__(self, path: str): self.path = path def __enter__(self): self.connection = sqlite3.connect(self.path) self.cursor = self.connection.cursor() return self def __exit__(self, exc_type, exc_val, exc_tb): if exc_type is not None: print(f'an error occurred: {exc_val}') self.connection.close()

You can use this context manager in conjunction with the

with

statement in your code as follows:


import sqlite3 create_table_sql_statement = 'CREATE TABLE IF NOT EXISTS books(title TEXT, author TEXT)' insert_into_table_sql_statement = "INSERT INTO books VALUES ('If Tomorrow Comes', 'Sidney Sheldon'), ('The Lincoln Lawyer', 'Michael Connelly')" select_from_table_sql_statement = 'SELECT * FROM books' class Database: def __init__(self, path: str): self.path = path def __enter__(self): self.connection = sqlite3.connect(self.path) self.cursor = self.connection.cursor() return self def __exit__(self, exc_type, exc_val, exc_tb): if exc_type is not None: print(f'an error occurred: {exc_val}') def main(): with Database(':memory:') as db: db.cursor.execute(create_table_sql_statement) db.connection.commit() db.cursor.execute(insert_into_table_sql_statement) db.connection.commit() db.cursor.execute(select_from_table_sql_statement) print(db.cursor.fetchall()) if __name__ == '__main__': main() # [('If Tomorrow Comes', 'Sidney Sheldon'), ('The Lincoln Lawyer', 'Michael Connelly')]

Evident from the output of the

print()

function call, the program has successfully stored and retrieved the given data from the database.

Without the

with

statement,

Database

is just a plain old class. However, the moment you put

with

infront of it, the three methods hop into action.

The

__init__()

method is the initializer and works identically to any other plain Python class’s initializer method. It takes the path to the database.

The

__enter__()

method sets up the connection to the database and returns the instance of the context manager class to the target variable,

db

in this case.

This target variable is now encapsulating both the connection and the cursor objects. You can access them as

db.connection

and

db.cursor

respectively.

Once the code inside the

with

block finishes running, the

__exit__()

method will execute and close the active connection to the database.

You can handle any exception that may occur during the execution inside the

__exit__()

method. If there is an exception,

exc_type

holds the type of the exception,

exc_val

holds the value of the exception,

exc_tb

holds the traceback.

If there is no exception, the three variables will have a value of

None

. I’ll not get into the details of exception handling in this article since that can take on many forms depending on what you’re dealing with.

To make this custom context manager accessible from anywhere in the program, you can put it into its own separate module or even package.

This is far better solution than the

try...except...finally

ladder you saw earlier. You don’t have to repeat yourself and chances of a human error is lower.

How to Create a Generator Based Context Manager in Python

Evident from the title of this section, this approach uses a generator instead of a class to implement a context manager.

Syntactically, generators are almost the same as normal functions, except that you need to use

yield

instead of

return

in a generator.

Writing a generator-based context manager requires less code but it also loses some of its readability.

You can write the generator-based equivalent of the class-based

Database

context manager as follows:


import sqlite3 from contextlib import contextmanager @contextmanager def database(path: str): connection = sqlite3.connect(path) try: cursor = connection.cursor() yield {'connection': connection, 'cursor': cursor} except Exception as e: print(f'an error occurred: {e}') finally: connection.close()

Instead of a class, you have a generator function here so there is no initializer. Instead, the function itself can accept the path to the database as a parameter.

Within a

try

block, you can establish a connection to the database, instantiate the cursor, and return both objects to the user.

You can write

yield connection, cursor

to return the two objects but in that case the generator will return them as a tuple.

I prefer to use strings over numbers as accessors and that’s why I have put the two objects inside a dictionary with descriptive keys.

The

except

block will run in case of an exception. Feel free to implement any exception handling strategy that you see fit.

The

finally

block will run unconditionally and close the open connection at the end of the

with

block.

Since there are no

__enter__()

or

__exit__()

methods either, you need to decorate the generator with the

@contextmanager

decorator.

This decorator defines a factory function for

with

statement context managers, without needing to create a class or separate

__enter__()

and

__exit__()

methods.

Usage of this context manager is identical to its class-based conterpart except the capitalization of its name.


import sqlite3 from contextlib import contextmanager create_table_sql_statement = 'CREATE TABLE IF NOT EXISTS books(title TEXT, author TEXT)' insert_into_table_sql_statement = "INSERT INTO books VALUES ('If Tomorrow Comes', 'Sidney Sheldon'), ('The Lincoln Lawyer', 'Michael Connelly')" select_from_table_sql_statement = 'SELECT * FROM books' @contextmanager def database(path: str): connection = sqlite3.connect(path) try: cursor = connection.cursor() yield {'connection': connection, 'cursor': cursor} except Exception as e: print(f'an error occurred: {e}') finally: connection.close() def main(): database_path = ':memory:' with database(database_path) as db: db.get('cursor').execute(create_table_sql_statement) db.get('connection').commit() db.get('cursor').execute(insert_into_table_sql_statement) db.get('connection').commit() db.get('cursor').execute(select_from_table_sql_statement) print(db.get('cursor').fetchall()) if __name__ == '__main__': main() # [('If Tomorrow Comes', 'Sidney Sheldon'), ('The Lincoln Lawyer', 'Michael Connelly')]

Since

db

is a dictionary instead of an object in this case, you will need to use square braces or the

get()

method to access the connection or cursor object.

Python3


class


ContextManager():


def


__init__(


self


):


print


'init method called'


def


__enter__(


self


):


print


'enter method called'


return


self


def


__exit__(


self


, exc_type, exc_value, exc_traceback):


print


'exit method called'


with ContextManager() as manager:


print


'with statement block'

Output:

init method called
enter method called
with statement block
exit method called

In this case, a ContextManager object is created. This is assigned to the variable after the keyword i.e manager. On running the above program, the following get executed in sequence:

  • __init__()
  • __enter__()
  • statement body (code inside the with block)
  • __exit__()[the parameters in this method are used to manage exceptions]

File management using context manager: Let’s apply the above concept to create a class that helps in file resource management. The FileManager class helps in opening a file, writing/reading contents, and then closing it.

Python - Class Based Context Manager
Python – Class Based Context Manager

27.Implementing a Context Manager as a Generator¶

We can also implement Context Managers using decorators and generators. Python has a contextlib module for this very purpose. Instead of a class, we can implement a Context Manager using a generator function. Let’s see a basic, useless example:

from contextlib import contextmanager @contextmanager def open_file(name): f = open(name, ‘w’) try: yield f finally: f.close()

Okay! This way of implementing Context Managers appear to be more intuitive and easy. However, this method requires some knowledge about generators, yield and decorators. In this example we have not caught any exceptions which might occur. It works in mostly the same way as the previous method.

Let’s dissect this method a little.

  1. Python encounters the

    yield

    keyword. Due to this it creates a generator instead of a normal function.
  2. Due to the decoration, contextmanager is called with the function
    name (

    open_file

    ) as its argument.
  3. The

    contextmanager

    decorator returns the generator wrapped by the

    GeneratorContextManager

    object.
  4. The

    GeneratorContextManager

    is assigned to the

    open_file

    function. Therefore, when we later call the

    open_file

    function, we are actually calling the

    GeneratorContextManager

    object.

So now that we know all this, we can use the newly generated Context Manager like this:

with open_file(‘some_file’) as f: f.write(‘hola!’)

Watch Now This tutorial has a related video course created by the Real Python team. Watch it together with the written tutorial to deepen your understanding: Context Managers and Python’s with Statement

The

with

statement in Python is a quite useful tool for properly managing external resources in your programs. It allows you to take advantage of existing context managers to automatically handle the setup and teardown phases whenever you’re dealing with external resources or with operations that require those phases.

Besides, the context management protocol allows you to create your own context managers so you can customize the way you deal with system resources. So, what’s the

with

statement good for?

In this tutorial, you’ll learn:

  • What the Python

    with

    statement is for and how to use it
  • What the context management protocol is
  • How to implement your own context managers

With this knowledge, you’ll write more expressive code and avoid resource leaks in your programs. The

with

statement helps you implement some common resource management patterns by abstracting their functionality and allowing them to be factored out and reused.

Free Download: Get a sample chapter from Python Tricks: The Book that shows you Python’s best practices with simple examples you can apply instantly to write more beautiful + Pythonic code.

Take the Quiz: Test your knowledge with our interactive “Context Managers and Python’s with Statement” quiz. Upon completion you will receive a score so you can track your learning progress over time:

Python3


with


open


("test.txt") as f:


data


f.read()

Let’s take the example of file management. When a file is opened, a file descriptor is consumed which is a limited resource. Only a certain number of files can be opened by a process at a time. The following program demonstrates it.

Context Managers in Python - Advanced Python 21 - Programming Tutorial
Context Managers in Python – Advanced Python 21 – Programming Tutorial

Keywords searched by users: python class context manager

Python - Class Based Context Manager - Youtube
Python – Class Based Context Manager – Youtube
Python Tutorial: Context Managers - Efficiently Managing Resources - Youtube
Python Tutorial: Context Managers – Efficiently Managing Resources – Youtube
Python Timer Functions: Three Ways To Monitor Your Code – Real Python
Python Timer Functions: Three Ways To Monitor Your Code – Real Python
Understanding
Understanding “With” And Python’S Context Managers – Youtube
What Is A Context Manager? - Youtube
What Is A Context Manager? – Youtube

See more here: kientrucannam.vn

Trả lời

Email của bạn sẽ không được hiển thị công khai. Các trường bắt buộc được đánh dấu *