Python Class Context Manager | Python Context Manager Protocol

Using the async with Statement

The
with
statement also has an asynchronous version,
async with
. You can use it to write context managers that depend on asynchronous code. It’s quite common to see
async with
in that kind of code, as many IO operations involve setup and teardown phases.

For example, say you need to code an asynchronous function to check if a given site is online. To do that, you can use
aiohttp
,
asyncio
, and
async with
like this:

1# site_checker_v0.py 2 3import aiohttp 4import asyncio 5 6async def check(url): 7 async with aiohttp.ClientSession() as session: 8 async with session.get(url) as response: 9 print(f"{url}: status -> {response.status}") 10 html = await response.text() 11 print(f"{url}: type -> {html[:17].strip()}") 12 13async def main(): 14 await asyncio.gather( 15 check("https://realpython.com"), 16 check("https://pycoders.com"), 17 ) 18 19asyncio.run(main())

Here’s what this script does:

Line 3 imports
aiohttp
, which provides an asynchronous HTTP client and server for
asyncio
and Python. Note that
aiohttp
is a third-party package that you can install by running
python -m pip install aiohttp
on your command line.
Line 4 imports
asyncio
, which allows you to write concurrent code using the
async
and
await
syntax.
Line 6 defines
check()
as an asynchronous function using the
async
keyword.

Inside
check()
, you define two nested
async with
statements:

Line 7 defines an outer
async with
that instantiates
aiohttp.ClientSession()
to get a context manager. It stores the returned object in
session
.
Line 8 defines an inner
async with
statement that calls
.get()
on
session
using
url
as an argument. This creates a second context manager and returns a
response
.
Line 9 prints the response status code for the
url
at hand.
Line 10 runs an awaitable call to
.text()
on
response
and stores the result in
html
.
Line 11 prints the site
url
and its document type,
doctype
.
Line 13 defines the script’s
main()
function, which is also a coroutine.
Line 14 calls
gather()
from
asyncio
. This function runs awaitable objects in a sequence concurrently. In this example,
gather()
runs two instances of
check()
with a different URL for each.
Line 19 runs
main()
using
asyncio.run()
. This function creates a new
asyncio
event loop and closes it at the end of the operation.

If you run this script from your command line, then you get an output similar to the following:

$ python site_checker_v0.py https://realpython.com: status -> 200 https://pycoders.com: status -> 200 https://pycoders.com: type ->
https://realpython.com: type ->

Cool! Your script works and you confirm that both sites are currently available. You also retrieve the information regarding document type from each site’s home page.

Note: Your output can look slightly different due to the nondeterministic nature of concurrent task scheduling and network latency. In particular, the individual lines can come out in a different order.

The
async with
statement works similar to the regular
with
statement, but it requires an asynchronous context manager. In other words, it needs a context manager that is able to suspend execution in its enter and exit methods. Asynchronous context managers implement the special methods
.__aenter__()
and
.__aexit__()
, which correspond to
.__enter__()
and
.__exit__()
in a regular context manager.

The
async with ctx_mgr
construct implicitly uses
await ctx_mgr.__aenter__()
when entering the context and
await ctx_mgr.__aexit__()
when exiting it. This achieves
async
context manager behavior seamlessly.

Managing Resources in Python

One common problem you’ll face in programming is how to properly manage external resources, such as files, locks, and network connections. Sometimes, a program will retain those resources forever, even if you no longer need them. This kind of issue is called a memory leak because the available memory gets reduced every time you create and open a new instance of a given resource without closing an existing one.

Managing resources properly is often a tricky problem. It requires both a setup phase and a teardown phase. The latter phase requires you to perform some cleanup actions, such as closing a file, releasing a lock, or closing a network connection. If you forget to perform these cleanup actions, then your application keeps the resource alive. This might compromise valuable system resources, such as memory and network bandwidth.

For example, a common problem that can arise when developers are working with databases is when a program keeps creating new connections without releasing or reusing them. In that case, the database back end can stop accepting new connections. This might require an admin to log in and manually kill those stale connections to make the database usable again.

Another frequent issue shows up when developers are working with files. Writing text to files is usually a buffered operation. This means that calling
.write()
on a file won’t immediately result in writing text to the physical file but to a temporary buffer. Sometimes, when the buffer isn’t full and developers forget to call
.close()
, part of the data can be lost forever.

Another possibility is that your application runs into errors or exceptions that cause the control flow to bypass the code responsible for releasing the resource at hand. Here’s an example in which you use
open()
to write some text to a file:

file = open("hello.txt", "w") file.write("Hello, World!") file.close()

This implementation doesn’t guarantee the file will be closed if an exception occurs during the
.write()
call. In this case, the code will never call
.close()
, and therefore your program might leak a file descriptor.

In Python, you can use two general approaches to deal with resource management. You can wrap your code in:

A
try
…
finally
construct
A
with
construct

The first approach is quite general and allows you to provide setup and teardown code to manage any kind of resource. However, it’s a little bit verbose. Also, what if you forget any cleanup actions?

The second approach provides a straightforward way to provide and reuse setup and teardown code. In this case, you’ll have the limitation that the
with
statement only works with context managers. In the next two sections, you’ll learn how to use both approaches in your code.

The try … finally Approach

Working with files is probably the most common example of resource management in programming. In Python, you can use a
try
…
finally
statement to handle opening and closing files properly:

# Safely open the file file = open("hello.txt", "w") try: file.write("Hello, World!") finally: # Make sure to close the file after using it file.close()

In this example, you need to safely open the file
hello.txt
, which you can do by wrapping the call to
open()
in a
try
…
except
statement. Later, when you try to write to
file
, the
finally
clause will guarantee that
file
is properly closed, even if an exception occurs during the call to
.write()
in the
try
clause. You can use this pattern to handle setup and teardown logic when you’re managing external resources in Python.

The
try
block in the above example can potentially raise exceptions, such as
AttributeError
or
NameError
. You can handle those exceptions in an
except
clause like this:

# Safely open the file file = open("hello.txt", "w") try: file.write("Hello, World!") except Exception as e: print(f"An error occurred while writing to the file: {e}") finally: # Make sure to close the file after using it file.close()

In this example, you catch any potential exceptions that can occur while writing to the file. In real-life situations, you should use a specific exception type instead of the general
Exception
to prevent unknown errors from passing silently.

The with Statement Approach

The Python
with
statement creates a runtime context that allows you to run a group of statements under the control of a context manager. PEP 343 added the
with
statement to make it possible to factor out standard use cases of the
try
…
finally
statement.

Compared to traditional
try
…
finally
constructs, the
with
statement can make your code clearer, safer, and reusable. Many classes in the standard library support the
with
statement. A classic example of this is
open()
, which allows you to work with file objects using
with
.

To write a
with
statement, you need to use the following general syntax:

with expression as target_var: do_something(target_var)

The context manager object results from evaluating the
expression
after
with
. In other words,
expression
must return an object that implements the context management protocol. This protocol consists of two special methods:

.__enter__()
is called by the
with
statement to enter the runtime context.
.__exit__()
is called when the execution leaves the
with
code block.

The
as
specifier is optional. If you provide a
target_var
with
as
, then the return value of calling
.__enter__()
on the context manager object is bound to that variable.

Note: Some context managers return
None
from
.__enter__()
because they have no useful object to give back to the caller. In these cases, specifying a
target_var
makes no sense.

Here’s how the
with
statement proceeds when Python runs into it:

Call
expression
to obtain a context manager.
Store the context manager’s
.__enter__()
and
.__exit__()
methods for later use.
Call
.__enter__()
on the context manager and bind its return value to
target_var
if provided.
Execute the
with
code block.
Call
.__exit__()
on the context manager when the
with
code block finishes.

In this case,
.__enter__()
, typically provides the setup code. The
with
statement is a compound statement that starts a code block, like a conditional statement or a
for
loop. Inside this code block, you can run several statements. Typically, you use the
with
code block to manipulate
target_var
if applicable.

Once the
with
code block finishes,
.__exit__()
gets called. This method typically provides the teardown logic or cleanup code, such as calling
.close()
on an open file object. That’s why the
with
statement is so useful. It makes properly acquiring and releasing resources a breeze.

Here’s how to open your
hello.txt
file for writing using the
with
statement:

with open("hello.txt", mode="w") as file: file.write("Hello, World!")

When you run this
with
statement,
open()
returns an
io.TextIOBase
object. This object is also a context manager, so the
with
statement calls
.__enter__()
and assigns its return value to
file
. Then you can manipulate the file inside the
with
code block. When the block ends,
.__exit__()
automatically gets called and closes the file for you, even if an exception is raised inside the
with
block.

This
with
construct is shorter than its
try
…
finally
alternative, but it’s also less general, as you already saw. You can only use the
with
statement with objects that support the context management protocol, whereas
try
…
finally
allows you to perform cleanup actions for arbitrary objects without the need for supporting the context management protocol.

In Python 3.1 and later, the
with
statement supports multiple context managers. You can supply any number of context managers separated by commas:

with A() as a, B() as b: pass

This works like nested
with
statements but without nesting. This might be useful when you need to open two files at a time, the first for reading and the second for writing:

with open("input.txt") as in_file, open("output.txt", "w") as out_file: # Read content from input.txt # Transform the content # Write the transformed content to output.txt pass

In this example, you can add code for reading and transforming the content of
input.txt
. Then you write the final result to
output.txt
in the same code block.

Using multiple context managers in a single
with
has a drawback, though. If you use this feature, then you’ll probably break your line length limit. To work around this, you need to use backslashes () for line continuation, so you might end up with an ugly final result.

The
with
statement can make the code that deals with system resources more readable, reusable, and concise, not to mention safer. It helps avoid bugs and leaks by making it almost impossible to forget cleaning up, closing, and releasing a resource after you’re done with it.

Using
with
allows you to abstract away most of the resource handling logic. Instead of having to write an explicit
try
…
finally
statement with setup and teardown code each time,
with
takes care of that for you and avoids repetition.

What is a Context Manager in Python?

According to the Python glossary, a context manager is —

An object which controls the environment seen in a
with
statement by defining
__enter__()
and
__exit__()
methods.

That may not be noticeably clear to you. Let me explain the concept with an example.

The
with
statement in Python lets you run a block of code within a runtime context defined by a context manager object.

Once the block of code has finished executing, the context manager object will take care of tearing down any external resources that are no longer needed.

You can rewrite the program by using the
with
statement as follows:

def main(): with open('books.txt', 'w') as my_file: my_file.write('If Tomorrow Comes by Sidney Sheldon') if __name__ == '__main__': main()

Since the
open()
function is paired with a
with
statement in this example, the function will create a context manager.

The file object will be accessible within the context of the indented code block, which means the file object doesn’t exist outside of that scope.

The
as
keyword is useful when you want to assign a target variable to a returned object. Here, the
my_file
variable is the target and will hold the file object.

You can do whatever you want within the indented block of code and don’t have to worry about closing the file.

Because once the block of code has finished executing the context manager will close the file automatically.

So, you have rewritten the entire
try...except...finally
ladder within two lines of code using the
with
statement and a context manager.

But how does that happen? How does a context manager object handle the task of setting up and closing resources?

And where are those
__enter__()
and
__exit__()
methods you read about on the Python documentation glossary?

Well, I’m so glad you asked 🙂

27.Handling Exceptions¶

We did not talk about the
type
,
value
and
traceback
arguments of the
__exit__
method. Between the 4th and 6th step, if
an exception occurs, Python passes the type, value and traceback of the
exception to the
__exit__
method. It allows the
__exit__
method
to decide how to close the file and if any further steps are required.
In our case we are not paying any attention to them.

What if our file object raises an exception? We might be trying to access a method on the file object which it does not supports. For instance:

with File(‘demo.txt’, ‘w’) as opened_file: opened_file.undefined_function(‘Hola!’)

Let’s list the steps which are taken by the
with
statement when
an error is encountered:

It passes the type, value and traceback of the error to the
__exit__
method.
It allows the
__exit__
method to handle the exception.
If
__exit__
returns
True
then the exception was gracefully handled.
If anything other than
True
is returned by the
__exit__
method then the exception is raised by the
with
statement.

In our case the
__exit__
method returns
None
(when no return
statement is encountered then the method returns
None
). Therefore,
the
with
statement raises the exception:

Traceback (most recent call last): File ”

“, line 2, in

AttributeError: ‘file’ object has no attribute ‘undefined_function’

Let’s try handling the exception in the
__exit__
method:

class File(object): def __init__(self, file_name, method): self.file_obj = open(file_name, method) def __enter__(self): return self.file_obj def __exit__(self, type, value, traceback): print(“Exception has been handled”) self.file_obj.close() return True with File(‘demo.txt’, ‘w’) as opened_file: opened_file.undefined_function() # Output: Exception has been handled

Our
__exit__
method returned
True
, therefore no exception was raised
by the
with
statement.

This is not the only way to implement Context Managers. There is another way and we will be looking at it in the next section.

Python Tutorial: Context Managers - Efficiently Managing Resources — Python Tutorial: Context Managers – Efficiently Managing Resources

Why use a context manager

Context managers keep our codebases much cleaner because they encapsulate administrative boilerplate and separate it from the business logic.

Additionally, context managers are structured to carry out their exit methods regardless of what happens in the code block they frame. So even if something goes wrong in the managed block, the context manager ensures the deallocations are performed and the default settings are restored.

Let’s give a solid example. Think about operating on a file without using
with
, like in the following block.

The first thing to note is that we must always close an open file. The
finally
block would perform the close even if an error occurred. If we had to do this try-except-finally logic every time we wanted to work with a file we’d have a lot of duplicate code.

Luckily, Python’s built-in
open()
is a context manager. Therefore, using a
with
statement, we can program the same logic like this:

Here,
open()
‘s enter method opens the file and returns a file object. The
as
keyword binds the returned value to , and we use to read the contents of
random.txt
. At the end of the execution of the inner code block, the exit method runs and closes the file.

We can check whether is actually closed (
with
does not define a variable scope, we can access the variables it created from outside the statement).

It’s evident from this simple example that context managers allow us to make our code cleaner and more reusable.

Python defines several other context managers in the standard library, but it also allows programmers to define context managers of their own.

In the next section, we will work on defining custom context managers. We will first work on the simple function-based implementation and later move on to the slightly more complicated class-based definitions.

Summary

A context manager, used in a
with
statement, defines a temporary context for the given set of operations. It does so by injecting code at the beginning and at the end of the code block, setting up the context at the beginning, and tearing it down at the end.

Don’t have an account yet? Sign up here.

How can you create your own context manager in Python?

A context manager is an object that can be used in a
with
block to sandwich some code between an entrance action and an exit action.

File objects can be used as context managers to automatically close the file when we’re done working with it:

>>> with open("example.txt", "w") as file: ... file.write("Hello, world!") ... 13 >>> file.closed True

Context managers need a
__enter__
method and a
__exit__
method, and the
__exit__
method should accept three positional arguments:

class Example: def __enter__(self): print("enter") def __exit__(self, exc_type, exc_val, exc_tb): print("exit")

This context manager just prints
enter
when the
with
block is entered and
exit
when the
with
block is exited:

>>> with Example(): ... print("Yay Python!") ... enter Yay Python! exit

Of course, this is a somewhat silly context manager. Let’s look at a context manager that actually does something a little bit useful.

This context manager temporarily changes the value of an environment variable:

import os class set_env_var: def __init__(self, var_name, new_value): self.var_name = var_name self.new_value = new_value def __enter__(self): self.original_value = os.environ.get(self.var_name) os.environ[self.var_name] = self.new_value def __exit__(self, exc_type, exc_val, exc_tb): if self.original_value is None: del os.environ[self.var_name] else: os.environ[self.var_name] = self.original_value

The
USER
environment variable on my machine currently has the value of
Trey
:

>>> print("USER env var is", os.environ["USER"]) USER env var is trey

If we use this context manager, within its
with
block, the
USER
environment variable will have a different value:

>>> with set_env_var("USER", "akin"): ... print("USER env var is", os.environ["USER"]) ... USER env var is akin

But after the
with
block exits, the value of that environment variable resets back to its original value:

>>> print("USER env var is", os.environ["USER"]) USER env var is trey

This is all thanks to our context manager’s
__enter__
method and a
__exit__
method, which run when our context manager’s
with
block is entered and exited.

askeyword?

You’ll sometimes see context managers used with an
as
keyword (note the
as result
below):

>>> with set_env_var("USER", "akin") as result: ... print("USER env var is", os.environ["USER"]) ... print("Result from __enter__ method:", result) ...

The
as
keyword will point a given variable name to the return value from the
__enter__
method:

In our case, we always get
None
as the value of our
result
variable:

>>> with set_env_var("USER", "akin") as result: ... print("USER env var is", os.environ["USER"]) ... print("Result from __enter__ method:", result) ... USER env var is akin Result from __enter__ method: None

This is because our
__enter__
method doesn’t return anything, so it implicitly returns the default function return value of
None
.

__enter__

Let’s look at a context manager that does return something from
__enter__
.

Here we have a program called
timer.py
:

import time class Timer: def __enter__(self): self.start = time.perf_counter() return self def __exit__(self, exc_type, exc_val, exc_tb): self.stop = time.perf_counter() self.elapsed = self.stop - self.start

This context manager will time how long it took to run a particular block of code (the block of code in our
with
block).

We can use this context manager by making a
Timer
object, using
with
to run a block of code, and then checking the
elapsed
attribute on our
Timer
object:

>>> t = Timer() >>> with t: ... result = sum(range(10_000_000)) ... >>> t.elapsed 0.28711878502508625

But there’s actually an even shorter way to use this context manager.

We can make the
Timer
object and assign it to a variable, all on one line of code, using our
with
block and the
as
keyword:

>>> with Timer() as t: ... result = sum(range(10_000_000)) ... >>> t.elapsed 0.3115791230229661

This works because our context manager’s
__enter__
method returns
self
:

def __enter__(self): self.start = time.perf_counter() return self

So it’s returning the actual context manager object to us and that’s what gets assigned to the variable in our
with
block.

Since many context managers keep track of some useful state on their own object, it’s very common to see a context manager’s
__enter__
method return
self
.

__exit__

What about that
__exit__
method?

def __exit__(self, exc_type, exc_val, exc_tb): self.stop = time.perf_counter() self.elapsed = self.stop - self.start

What are those three arguments that it accepts? And does its return value matter?

If an exception occurs within a
with
block, these three arguments passed to the context manager’s
__exit__
method will be:

But if no exception occurs, those three arguments will all be
None
.

Here’s a context manager that uses all three of those arguments:

import logging class LogException: def __init__(self, logger, level=logging.ERROR, suppress=False): self.logger, self.level, self.suppress = logger, level, suppress def __enter__(self): return self def __exit__(self, exc_type, exc_val, exc_tb): if exc_type is not None: info = (exc_type, exc_val, exc_tb) self.logger.log(self.level, "Exception occurred", exc_info=info) return self.suppress return False

This context manager logs exceptions as they occur (using Python’s
logging
module).

So we can use this
LogException
context manager like this:

import logging from log_exception import LogException logging.basicConfig(level=logging.DEBUG) logger = logging.getLogger("example") with LogException(logger): result = 1 / 0 # This will cause a ZeroDivisionError print("That's the end of our program")

When an exception occurs in our code, we’ll see the exception logged to our console:

$ python3 log_example.py ERROR:example:Exception occurred Traceback (most recent call last): File "/home/trey/_/log_example.py", line 8, in result = 1 / 0 # This will cause a ZeroDivisionError ~~^~~ ZeroDivisionError: division by zero Traceback (most recent call last): File "/home/trey/_/log_example.py", line 8, in result = 1 / 0 # This will cause a ZeroDivisionError ~~^~~ ZeroDivisionError: division by zero

We see
ERROR
, the name of our logger (
example
),
Exception occurred
, and then the traceback.

In this example, we also a second traceback, which was printed by Python when our program crashed.

Because our program exited, it didn’t actually print out the last line in our program (
That's the end of our program
).

__exit__

If we had passed
suppress=True
to our context manager, we’ll see something different happen:

import logging from log_exception import LogException logging.basicConfig(level=logging.DEBUG) logger = logging.getLogger("example") with LogException(logger, suppress=True): result = 1 / 0 # This will cause a ZeroDivisionError print("That's the end of our program")

Now when we run our program, the exception is logged, but then our program continues onward after the
with
block:

$ python3 log_example.py ERROR:example:Exception occurred Traceback (most recent call last): File "/home/trey/_/_/log_example.py", line 8, in result = 1 / 0 # This will cause a ZeroDivisionError ~~^~~ ZeroDivisionError: division by zero That's the end of our program

We can see
That's the end of our program
actually prints out here!

What’s going on?

So this
suppress
argument, it’s used by our context manager to suppress an exception:

import logging class LogException: ... def __exit__(self, exc_type, exc_val, exc_tb): if exc_type is not None: info = (exc_type, exc_val, exc_tb) self.logger.log(self.level, "Exception occurred", exc_info=info) return self.suppress return False

If the
__exit__
method returns something true or truthy, whatever exception was being raised will actually be suppressed.

By default,
__exit__
returns
None
, just as every function does by default.
If we return
None
, which is falsey, or
False
, or anything that’s falsey,
__exit__
won’t do anything different from its default, which is to just continue raising that exception.

But if
True
or a truthy value is returned, the exception will be suppressed.

contextmanager?

Have you ever seen a generator function that somehow made a context manager?

Python’s
contextlib
module includes a decorator which allows for creating context managers using a function syntax (instead of using the typical class syntax we saw above):

from contextlib import contextmanager import os @contextmanager def set_env_var(var_name, new_value): original_value = os.environ.get(var_name) os.environ[var_name] = new_value try: yield finally: if original_value is None: del os.environ[var_name] else: os.environ[var_name] = original_value

Interestingly, this fancy decorator still involves
__enter__
and
__exit__
under the hood: it’s just a very clever helper for creating an object that has those methods.

This
contextmanager
decorator can sometimes be very handy, though it does have limitations!
I plan to record a separate screencast (and write a separate article) on
contextlib.contextmanager
.
Python Morsels subscribers will hear about this new screencast as soon as I publish it. 😉

__enter__&
__exit__

Context managers are objects that work in a
with
block.

You can make a context manager by creating an object that has a
__enter__
method and a
__exit__
method.

Python also includes a fancy decorator for creating context managers with a function syntax, which I’ll cover in a future screencast.

Need to fill-in gaps in your Python skills?

Need to fill-in gaps in your Python skills? I send weekly emails designed to do just that.

Summary: in this tutorial, you’ll learn about the Python context managers and how to use them effectively

Context Managers in Python Make Life Easier

Summary

Use Python context managers to define runtime contexts when executing in the
with
statement.
implement the
__enter__()
and
__exit__()
methods to support the context manager protocol.

When we watch youtube videos or any tutorial about data analysis, we see that the context manager is mainly used in reading or writing some texts from a file so that we don’t have to write the code for closing the file whenever our task is finished. But this is not the only use-case of this context manager. We can use context managers in more relaxed ways. For example, we can easily interchange the working directory using a context manager. We can also access a database efficiently by this. In this article, we will discuss everything about it and see some examples that will help us use them properly in your project. So what are we waiting for? Let’s get started.

This article was published as a part of the Data Science Blogathon.

A context manager is actually a resource manager in python. For example, when we read or write some content from a file, we don’t care about closing the file. If we are using one file, then this is not a big issue. But when it comes to multiple files, this will cause problems as we are not releasing the resources we are no longer using. This is also true for database access. After a database is used, we should close the database. Otherwise, it can slow down our code. So, the context manager helps us maintain the resource adequately. Below is an example of a context manager used for accessing the file.

with open(“file.txt”,”r”) as f: print(f.read())

Here we are trying to open the file in read mode using context manager and reading the first line from the file. The file will automatically close when we leave the context.

In the previous example, we used the open() function as a context manager. There are two ways to define a context manager – one is class-based and the other is function-based. Let’s see how we can create it using class.

Python Code:

For the class-based, we have to add two methods: enter () and exit(). In the enter() method, we define the things we want to do in the context. Here we open a file. In the exit() method, we have to write things that will happen after leaving the context. In this case, we will close the file after leaving the context. Now let’s see what a function-based context manager looks like.

import contextlib @contextlib.contextmanager def my_context(): # add any set up code you need yield # add any teardown code you need

For a function-based context manager, you define a function with the
@contextlib.contextmanager
decorator. Within that function, you place the code you want to execute within the context. Following that, you use the
yield
keyword to indicate that this function serves as a context manager. After
yield
, you can include your teardown code, specifying actions to take when exiting the context. This teardown code ensures proper resource cleanup and is executed when leaving the context.

We discussed the theoretical part of the context manager. Now it is time to see some examples. Let’s see how we can access the database.

import psycopg2 @contextlib.contextmanager def database(url): # set up database connection db = psycopg2.connect(url) yield db # tear down database connection db.disconnect()

You have already understood what is happening in the function. This context manager will give access to a database in its context. When we leave the context, the connection to the database is ended. You don’t have to return value whenever creating a context manager.

Now here is another example. While using Google Colab, if the data size is too large, we attach our Google Drive to the notebook and create a folder for the data. Then we download the data in that folder and want to return it to our working directory. This we can do easily with a context manager.

@contextlib.contextmanager def in _ dir(path): # save current working directory old _ dir = os.getcwd() # switch to new working directory os.chdir(path) yield # change back to previous # working directory os.chdir(old _ dir)

See the above example. We don’t return any value from the “yield” keyword. I hope you now understand how to use context manager.

Sometimes it happens that we need to open multiple files. So what should we do? Of course, we have to use multiple context managers. Here is an example of copying the content of a file.

with open(“file.txt”, “r”) as f: contents = f.read() with open(“copy.txt”, “w”) as cf: cf.write(contents)

This will work, but if we have a large file to copy, it will cause pain for us. That will make the code slower. You can also face device freezing as that process takes a lot of resources. So what should we do? If we can copy the file line by line, that doesn’t cause any problem. To do this, we have to open two files simultaneously. See the below example.

with open(“file.txt”, “r”) as f: with open(“copy.txt”, “w”) as cf: for line in f: cf.write(line)

This is called the nested context. This method is fine when you have to open two files or three files. But when the number of files is much more, the code becomes messier and you have to face difficulties while editing the code. In Python 3.10, they introduced a new method “parenthesized context” in which you can write multiple contexts in one line using brackets. No need to use looping.

with (open(“file.txt”, “r”) as f, open(“copy.txt”, “w”) as cf): for line in f: cf.write(line)

The above code is much cleaner. For me, this is the best option for working on multiple files. Now it may happen that you will prefer others. That’s totally up to you.

Using context manager in your code ensures the proper utilization of the resource. Resource utilization is not a big deal when you are working on a small project, but when it comes to large projects, like reading a very large file that we discussed previously, context manager comes in handy. In this article, we learned –

Thank you for reading this article till the last. I hope you enjoyed this article. If there is any issue, please leave a comment. And don’t forget to check out my other articles in AnalyticsVidhya.

A. A context manager in Python is an object that defines the methods __enter__() and __exit__(). It is used to set up and tear down resources, such as file handling or database connections, in a clean and controlled manner.

A. The @contextmanager decorator is used to create a context manager as a Python generator function. It simplifies the creation of context managers by yielding control to the code block defined within the generator and handling resource setup and teardown.

A. Context managers ensure proper resource management and clean-up, which is critical to prevent resource leaks and errors. They provide a convenient and safe way to work with resources like files, databases, and network connections.

A. Yes, a context manager in Python is a construct used for resource management and is crucial for ensuring that resources are acquired and released properly, even in the presence of exceptions or errors.

The media shown in this article is not owned by Analytics Vidhya and is used at the Author’s discretion.

Conclusion

The Python
with
statement is a powerful tool when it comes to managing external resources in your programs. Its use cases, however, aren’t limited to resource management. You can use the
with
statement along with existing and custom context managers to handle the setup and teardown phases of a given process or operation.

The underlying context management protocol allows you to create custom context managers and factor out the setup and teardown logic so you can reuse them in your code.

In this tutorial, you learned:

What the Python
with
statement is for and how to use it
What the context management protocol is
How to implement your own context managers

With this knowledge, you’ll write safe, concise, and expressive code. You’ll also avoid resource leaks in your programs.

Take the Quiz: Test your knowledge with our interactive “Context Managers and Python’s with Statement” quiz. Upon completion you will receive a score so you can track your learning progress over time:

Watch Now This tutorial has a related video course created by the Real Python team. Watch it together with the written tutorial to deepen your understanding: Context Managers and Python’s with Statement

Managing Resources: In any programming language, the usage of resources like file operations or database connections is very common. But these resources are limited in supply. Therefore, the main problem lies in making sure to release these resources after usage. If they are not released then it will lead to resource leakage and may cause the system to either slow down or crash. It would be very helpful if users have a mechanism for the automatic setup and teardown of resources. In Python, it can be achieved by the usage of context managers which facilitate the proper handling of resources. The most common way of performing file operations is by using the keyword as shown below:

Building A Custom Context Manager In Python: A Closer Look

Summarizing the with Statement’s Advantages

To summarize what you’ve learned so far, here’s an inexhaustive list of the general benefits of using the Python
with
statement in your code:

Makes resource management safer than its equivalent
try
…
finally
statements
Encapsulates standard uses of
try
…
finally
statements in context managers
Allows reusing the code that automatically manages the setup and teardown phases of a given operation
Helps avoid resource leaks

Using the
with
statement consistently can improve the general quality of your code and make it safer by preventing resource leak problems.

Python context manager – bạn đã thực sự hiểu?

Bài đăng này đã không được cập nhật trong 2 năm

Bài viết gốc: https://manhhomienbienthuy.github.io/2017/05/12/python-context-managers.html (đã xin phép tác giả

)

Trong Python, context manager là một phương thức cho phép bạn cấp phát và sử dụng tài nguyên một cách hiệu quả. Context manager được sử dụng rộng rãi thông qua câu lệnh
with
. Ví dụ:

with open('foo', 'w') as f: f.write('Hora! We opened this file')

Đoạn code trên mở một file, ghi dữ liệu và đóng file lại. Nếu có bất kỳ lỗi gì xảy ra, thì file cũng luôn luôn được đảm bảo là đã đóng. Đoạn code trên nếu viết mà không sử dụng context manager thì sẽ trông như dưới đây:

f = open('foo', 'w') try: f.write('Hora! We opened this file') finally: f.close()

So sánh hai cách viết này thì chúng ta đã thấy rất rõ ràng rằng, context manager cho chúng ta cách viết code ngắn gọn hơn hẳn. Lệnh
with
cho chúng ta bảo đảm rằng file luôn luôn được đóng mà không cần biết những logic xử lý bên trong.

Hẳn là các bạn đã rất quen thuộc với những đoạn code trên, đặc biệt khi bạn đã từng nghe nói đến “Idiomatic Python”. Nhưng liệu bạn đã chắc chắn hiểu được cách làm việc chính xác với file và lý do tại sao đó lại là cách đúng không? Hoặc đơn giản hơn, bạn có biết khi nào thì mình đã thao tác sai không.

Nếu câu trả lời là không, thì bài viết này chính là dành cho bạn.

Context manager thường được sử dụng để lock các tài nguyên (trường hợp mở và đóng file là một ví dụ kinh điển cho việc này).

Quản lý tài nguyên

Tính năng quan trọng nhất và cũng là phổ biến nhất của context manager là để quản lý tài nguyên một cách chính xác. Quay lại với việc đọc và ghi file ở ví dụ trên, tại sao chúng ta phải sử dụng context manager. Mỗi khi mở một file để đọc hoặc ghi, một tài nguyên của hệ thống, trong trường hợp này là file descriptor sẽ đã bị tiêu tốn để chúng ta có thể thao tác. Thật không may là tài nguyên này lại là hữu hạn. Mỗi hệ điều hành đều có giới hạn nhất định cho số lượng file có thể mở cùng một lúc.

Không tin ư, bạn hãy xem ví dụ sau:

>>> files = [] >>> for _ in range(10000): ... files.append(open('foo', 'w')) ... Traceback (most recent call last): File " ", line 2, in OSError: [Errno 24] Too many open files: 'foo'

Ngoài lề một chút, file descriptor thực chất là một số nguyên. Khi bạn mở một file, hệ điều hành sẽ tạo ra một entry (có thể ở kernel) lưu trữ những thông tin liên quan đến file được mở. Mỗi entry sẽ được gán với 1 số nguyên (và số này sẽ là duy nhất), cho phép người dùng thông qua đó để thao tác với file.

Thực chất người dùng đang thao tác một cách gián tiếp thông qua file descriptor (và thông qua entry của kernel) với các dữ liệu thật sự được ghi ở bộ nhớ ngoài. Việc này mang lại nhiều lợi ích như có thể chia sẻ file cho nhiều tiến trình khác nhau cũng như duy trì bảo mật cho các file đó.

Thực ra, hằng ngày bạn đều đang làm việc với file descriptor mà có thể bạn cũng không nhận ra. Hệ điều hành đã gán sẵn một số file descriptor như cho bàn phím, cho màn hình, v.v… Và mọi thao tác chúng ta làm với máy tính đều thông qua các file descriptor này. Bạn nghĩ sao về việc chuyển hướng của câu lệnh Linux như thế này:

$ sort < file_list > sorted_file_list 2>&1

Tương tự như vậy, khi bạn mở một socket, một socket descriptor cũng sẽ được sử dụng.

Quay trở lại với nội dung của bài viết. Vậy điều gì xảy khi code Python của bạn mở file mà không đóng nó lại. Rất hiển nhiên, chúng ta đã mất một file descriptor mà chúng ta sẽ không bao giờ cần đến nó nữa. Điều này đồng nghĩa với việc, số file chúng ta có thể thao tác sẽ ít dần đi, do số lượng của chúng bị giới hạn. Mà việc “quên” không đóng file nhiều khi xảy ra khá thường xuyên, và tích tiểu thành đại, đến một lúc nào đó bạn không thể mở thêm file nào nữa.

Bạn có thể dùng lệnh

ulimit -n
để kiểm tra xem hệ thống của mình cho phép mở tối đa bao nhiêu file cùng lúc.

Tất nhiên là mọi vấn đề đều có thể giải quyết được. Vẫn với ví dụ trên, chúng ta có thể xử lý bằng cách đóng từng file một như sau:

>>> files = [] >>> for _ in range(10000): ... f = open('foo', 'w') ... f.close() ... files.append(f) ... >>>

Quản lý tài nguyên hiệu quả hơn

Trên đây là một cách giải quyết tuy vẫn hoạt động tốt nhưng không được thông minh cho lắm. Trong những hệ thống phức tạp hơn, rất khó để đảm bảo rằng tất cả các file đã được đóng lại khi không dùng đến nữa.

Giả sử trong quá trình thao tác, chúng ta gặp phải một exception nào đó thì phải làm thế nào đây. Bắt exception và xử lý riêng sao? Phải bắt những exception nào mới thì gọi là đủ?

Hoặc kể cả không có exception nhưng hàm đã
return
trước khi file kịp close thì sao? Trong những trường hợp phức tạp như vậy, làm thế nào để chúng ta “nhớ” phải đóng file lại. Câu trả lời là khó tới gần như không thể (thật phũ phàng).

Trong nhiều ngôn ngữ, lập trình viên cần phải sử dụng cấu trúc kiểu như
try ... except ... finally ...
để đảm bảo rằng file sẽ được đóng. Rất may mắn, Python đã nghĩ đến những khó khăn này của chúng ta và đưa cho chúng ta một phương thức dễ dàng để làm những việc đó – context manager.

Nói một cách ngắn gọn, chúng ta cần một phương thức càng đơn giản càng tốt để đảm bảo các tài nguyên được dọn dẹp cẩn thận dù có xảy ra bất cứ chuyện gì đi chăng nữa. Và context manager sẽ cung cấp cho chúng ta tính năng này:

with something_that_returns_a_context_manager() as my_resource: do_something(my_resource) ... print('done using my_resource')

Đơn giản vậy đó. Bằng cách sử dụng
with
, chúng ta sẽ đưa mọi thứ vào trong một context manager. Chúng ta gán context manager này cho một biến, và biến đó chỉ tồn tại khi block sau đó được thực thi. Điều này giống như chúng ta tạo một hàm, nó sẽ gọi một số thao tác và khi kết thúc, nó sẽ tự dọn dẹp những gì nó tạo ra.

Một số context manager hữu ích khác

Context manager thực sự rất cần thiết trong Python, và nó đã có mặt trong thư việc chuẩn. Một số context manager có thể bạn đã từng làm việc là
zipfile.ZipFiles
,
subprocess.Popen
,
tarfile.TarFile
,
telnetlib.Telnet
,
pathlib.Path
, v.v… Thậm chí,
Lock
của
threading
cũng là context manager. Trên thực tế, tất cả những tài nguyên mà chúng ta cần
close
sau khi sử dụng đều (và rất nên) là context manager.

Việc sử dụng
Lock
tương đối đặc biệt một chút. Trong trường hợp này, tài nguyên là một mutex. Sử dụng context manager sẽ phòng tránh được deadlock trong lập trình multithread nếu chúng ta sử dụng khóa mà không bao giờ mở nó. Hãy xem xét ví dụ sau:

>>> from threading import Lock >>> lock = Lock() >>> def do_something_dangerous(): ... lock.acquire() ... raise Exception('OOPS! I forgot this code could raise exceptions') ... lock.release() ... >>> try: ... do_something_dangerous() ... except: ... print('Got an exception') ... Got an exception >>> lock.acquire()

Với code trên, rõ ràng là
lock.release()
sẽ không bao giờ được gọi, và do đó, mọi tiến trình sẽ gặp deadlock và chết cứng ở đó (
lock.acquire()
sẽ không bao giờ kết thúc). Rất may mắn, với context manager, điều này có thể sửa chữa được:

>>> from threading import Lock >>> lock = Lock() >>> def do_something_dangerous(): ... with lock: ... raise Exception('oops I forgot this code could raise exceptions') ... >>> try: ... do_something_dangerous() ... except: ... print('Got an exception') ... Got an exception >>> lock.acquire() True >>> print('We can get here') We can get here >>>

Trên thực tế, không có cách nào để gây ra deadlock nếu sử dụng context manager. Và đây là điều chúng ta đang cần.

Trong phần tiếp theo, chúng ta sẽ tìm hiểu cách cài đặt một context manager, và qua đó, chúng ta sẽ hiểu hơn về cách thức một context manager hoạt động.

Cài đặt context manager như một class

Có nhiều cách khác nhau để cài đặt một context manager. Cách đơn giản nhất là cài đặt một class với hai phương thức vô cùng đặc biệt:
__enter__
và
__exit__
. Phương thức
__enter__
sẽ trả về tài nguyên cần quản lý (ví dụ như file đang được mở) và
__exit__
sẽ làm việc dọn dẹp hệ thống.

Hãy xem xét ví dụ sau về một context manager khi làm việc với file:

>>> class File: ... def __init__(self, file_name, method): ... self.file_obj = open(file_name, method) ... def __enter__(self): ... return self.file_obj ... def __exit__(self, type, value, traceback): ... self.file_obj.close() ... >>>

Class trên cũng như nhiều class khác, phương thức
__init__
để để khởi tạo đối tượng, trong trường hợp này là khởi tạo tên file cần mở cùng với mode (đọc/ghi) của nó. Phương thức
__enter__
mở file và trả về đối tượng file để thao tác với file đó trong khi
__exit__
chỉ đơn giản là đóng file lại.

Với hai phương thức
__enter__
và
__exit__
, chúng ta có thể sử dụng class này cùng với
with
:

>>> with File('foo', 'w') as f: ... f.write('Hora! We opened this file') ... 25

Phương thức
__exit__
bắt buộc phải có 3 tham số. Dưới đây là những gì thực sự xảy ra khi chúng ta gọi context manager:

Câu lệnh
with
lưu phương thức
__exit__
của class
File
Câu lệnh này gọi phương thức
__enter__
của class
File
Phương thức
__enter__
mở file và trả về object để thao tác với file đó
Object được trả về được truyền cho biến
Chúng ta thao tác với file bằng cách ghi dữ liệu
f.write
Khi kết thúc block, câu lệnh
with
gọi phương thức
__exit__
Phương thức
__exit__
đóng file cho chúng ta

Xử lý exception

Trong cài đặt đơn giản trên, chúng ta đã bỏ qua 3 tham số
type
,
value
,
traceback
của phương thức
__exit__
. Tuy nhiên, trong quá trình thực thi block lệnh ở trên, nếu xảy ra một exception, Python sẽ chuyển những thông tin
type
,
value
và
traceback
của exception này tới phương thức
__exit__
. Điều đó giúp chúng ta có thể tùy biến phương thức
__exit__
để xử lý những vấn đề có thể xảy ra trong quá trình thực thi. Trong trường hợp của chúng ta, chúng ta chỉ cần đóng file và không cần quan tâm đến exception này.

Nhưng chuyện gì sẽ xảy ra nếu bản thân file object gặp phải một exception? Ví dụ, khi chúng ta thử gọi một phương thức không tồn tại:

>>> with File('foo', 'w') as f: ... f.undefined_function('Oops! I called an unknown method') ... Traceback (most recent call last): File " ", line 2, in AttributeError: '_io.TextIOWrapper' object has no attribute 'undefined_function'

Dưới đây là quy trình những gì đã xảy ra khi có lỗi xảy ra:

type
,
value
,
traceback
của lỗi đó được truyền cho
__exit__
Trong phương thức
__exit__
, chúng ta có thể tùy ý xử lý exception đó
Nếu
__exit__
trả về
True
thì exception đã được xử lý hoàn toàn.
Nếu không, exception sẽ tiếp tục được raise bởi lệnh
with

Trong trường hợp của chúng ta, phương thức
__exit__
không trả về bất cứ thứ gì, do đó, lệnh
with
sẽ raise exception.

Chúng ta có thể tạm xử lý exception như sau:

>>> class File: ... def __init__(self, file_name, method): ... self.file_obj = open(file_name, method) ... def __enter__(self): ... return self.file_obj ... def __exit__(self, type, value, traceback): ... print("Exception has been handled") ... self.file_obj.close() ... return True ... >>> with File('foo', 'w') as f: ... f.undefined_function() ... Exception has been handled >>>

Phương thức
__exit__
trả về
True
, do đó, không có exception nào được raise bởi lệnh
with
.

Có nhiều cách để cài đặt context manager. Trên đây là một cách đơn giản và dễ hiểu nhất. Trong phần tiếp theo, chúng ta sẽ tìm hiểu thêm một số phương pháp cài đặt nữa.

Sử dụng contextlib cài đặt context manager

Context manager quả là tiện lợi và hữu ích vô cùng. Do đó, trong thư viện chuẩn của Python có hẳn module
contextlib
với rất nhiều công cụ để tạo và làm việc với context manager.

Chúng ta có thể cài đặt context manager bằng cách sử dụng decorator và generator.
contextlib
cung cấp cho chúng ta decorator
@contextmanager
để decorate các hàm generator chỉ gọi
yield
đúng một lần duy nhất. Với decorator này, tất cả những gì diễn ra trước
yield
đều được coi là thao tác của phương thức
__enter__
. Những gì dễn ra sau đó được coi là của phương thức
__exit__
.

Hãy xem xét ví dụ của chúng ta về quản lý file khi dùng
contextlib
:

>>> from contextlib import contextmanager >>> @contextmanager ... def open_file(path, mode): ... f = open(path, mode) ... yield f ... f.close() ... >>> files = [] >>> for _ in range(10000): ... with open_file('foo', 'w') as f: ... files.append(f) ... >>> for f in files: ... if not f.closed: ... print('not closed') ... >>>

Như chúng ta đã thấy, việc cài đặt context manager đã ngắn gọn hơn rất nhiều. Chúng ta chỉ cần mở file,
yield
đối tượng đó và đóng nó lại. Mọi việc còn lại sẽ do decorator
@contextmanager
đảm nhiệm.

Và ví dụ thực tế cho thấy rằng các file của chúng ta đã được quản lý tốt, tất cả chúng đã được đóng lại đầy đủ. Tuy nhiên, cách thức cài đặt tiện lợi này yêu cầu chúng ta phải có chút hiểu biết về decorator, generator cũng như lệnh
yield
. Có thể tóm tắt quá trình tạo context manager trên như sau:

Python tìm thấy
yield
, hàm này là một generator chứ không phải hàm thông thường.
Với decorator
@contextmanager
, hàm
open_file
sẽ được truyền là tham số cho hàm
contextmanager
.
Hàm
contextmanager
trả về generator được bọc trong object của
GeneratorContextManager
.
Object
GeneratorContextManager
được gán cho hàm
open_file
. Do đó, khi chúng ta gọi hàm này, thực ra chúng ta đang làm việc với object
GeneratorContextManager
.

Python docs còn một ví dụ khác thú vị hơn:

>>> from contextlib import contextmanager >>> @contextmanager ... def tag(name): ... print("<%s>" % name) ... yield ... print(" " % name) ... >>> with tag('h1'): ... print('foo') ...
foo

Trong tất cả các trường hợp trên, chúng ta cũng không hề xử lý exception, vì vậy, context manager của chúng ta sẽ hoạt động giống như code đầu tiên.

Một công cụ tiện lợi khác của
contextlib
là
ContextDecorator
. Nó cho phép chúng ta cài đặt các context manager theo kiểu class. Nhưng với việc kế thừa từ class
ContextDecorator
, bạn có thể sử dụng context manager với lệnh
with
thông thường, hoặc sử dụng nó như một decorator dùng để decorate các hàm khác. Chúng ta có thể xem xét ví dụ sau (tương tự như ví dụ tag HTML ở trên):

>>> from contextlib import ContextDecorator >>> class tag(ContextDecorator): ... def __init__(self, name): ... self.name = name ... def __enter__(self): ... print('<%s>' % self.name) ... return self ... def __exit__(self, *exc): ... print(' ' % self.name) ... return False ... >>> with tag('h1'): ... print('this is not html') ...
this is not html >>> @tag('h1') ... def content(): ... print('this is another non-html content') ... >>> content() this is another non-html content
>>>

Kết luận

Bài viết trình bày những hiểu biết của tôi về context manager của Python, cách nó hoạt động và hỗ trợ cho chúng ta trong công việc lập trình. Như các bạn đã thấy, chúng ta có thể làm rất nhiều thứ với context manager. Mục đích cao nhất của nó thì không bao giờ thay đổi: quản lý hiệu quả các tài nguyên.

Chúng ta không chỉ có thể dùng context manager mà còn có thể tự cài đặt context manager cho riêng mình. Hãy sử dụng context manager và làm cho cuộc sống dễ chịu hơn.

Context Managers¶

Context managers allow you to allocate and release resources precisely
when you want to. The most widely used example of context managers is
the
with
statement. Suppose you have two related operations which
you’d like to execute as a pair, with a block of code in between.
Context managers allow you to do specifically that. For example:

with open(‘some_file’, ‘w’) as opened_file: opened_file.write(‘Hola!’)

The above code opens the file, writes some data to it and then closes it. If an error occurs while writing the data to the file, it tries to close it. The above code is equivalent to:

file = open(‘some_file’, ‘w’) try: file.write(‘Hola!’) finally: file.close()

While comparing it to the first example we can see that a lot of
boilerplate code is eliminated just by using
with
. The main
advantage of using a
with
statement is that it makes sure our file
is closed without paying attention to how the nested block exits.

A common use case of context managers is locking and unlocking resources and closing opened files (as I have already shown you).

Let’s see how we can implement our own Context Manager. This should allow us to understand exactly what’s going on behind the scenes.

Expert Python Tutorial #6 - Context Managers — Expert Python Tutorial #6 – Context Managers

Single use, reusable and reentrant context managers¶

Most context managers are written in a way that means they can only be
used effectively in a
with
statement once. These single use
context managers must be created afresh each time they’re used –
attempting to use them a second time will trigger an exception or
otherwise not work correctly.

This common limitation means that it is generally advisable to create
context managers directly in the header of the
with
statement
where they are used (as shown in all of the usage examples above).

Files are an example of effectively single use context managers, since
the first
with
statement will close the file, preventing any
further IO operations using that file object.

Context managers created using
contextmanager()
are also single use
context managers, and will complain about the underlying generator failing
to yield if an attempt is made to use them a second time:

>>> from contextlib import contextmanager >>> @contextmanager … def singleuse(): … print(“Before”) … yield … print(“After”) … >>> cm = singleuse() >>> with cm: … pass … Before After >>> with cm: … pass … Traceback (most recent call last): … RuntimeError: generator didn’t yield

Reentrant context managers¶

More sophisticated context managers may be “reentrant”. These context
managers can not only be used in multiple
with
statements,
but may also be used inside a
with
statement that is already
using the same context manager.

threading.RLock
is an example of a reentrant context manager, as are
suppress()
,
redirect_stdout()
, and
chdir()
. Here’s a very
simple example of reentrant use:

>>> from contextlib import redirect_stdout >>> from io import StringIO >>> stream = StringIO() >>> write_to_stream = redirect_stdout(stream) >>> with write_to_stream: … print(“This is written to the stream rather than stdout”) … with write_to_stream: … print(“This is also written to the stream”) … >>> print(“This is written directly to stdout”) This is written directly to stdout >>> print(stream.getvalue()) This is written to the stream rather than stdout This is also written to the stream

Real world examples of reentrancy are more likely to involve multiple functions calling each other and hence be far more complicated than this example.

Note also that being reentrant is not the same thing as being thread safe.
redirect_stdout()
, for example, is definitely not thread safe, as it
makes a global modification to the system state by binding
sys.stdout
to a different stream.

Reusable context managers¶

Distinct from both single use and reentrant context managers are “reusable” context managers (or, to be completely explicit, “reusable, but not reentrant” context managers, since reentrant context managers are also reusable). These context managers support being used multiple times, but will fail (or otherwise not work correctly) if the specific context manager instance has already been used in a containing with statement.

threading.Lock
is an example of a reusable, but not reentrant,
context manager (for a reentrant lock, it is necessary to use
threading.RLock
instead).

Another example of a reusable, but not reentrant, context manager is
ExitStack
, as it invokes all currently registered callbacks
when leaving any with statement, regardless of where those callbacks
were added:

>>> from contextlib import ExitStack >>> stack = ExitStack() >>> with stack: … stack.callback(print, “Callback: from first context”) … print(“Leaving first context”) … Leaving first context Callback: from first context >>> with stack: … stack.callback(print, “Callback: from second context”) … print(“Leaving second context”) … Leaving second context Callback: from second context >>> with stack: … stack.callback(print, “Callback: from outer context”) … with stack: … stack.callback(print, “Callback: from inner context”) … print(“Leaving inner context”) … print(“Leaving outer context”) … Leaving inner context Callback: from inner context Callback: from outer context Leaving outer context

As the output from the example shows, reusing a single stack object across multiple with statements works correctly, but attempting to nest them will cause the stack to be cleared at the end of the innermost with statement, which is unlikely to be desirable behaviour.

Using separate
ExitStack
instances instead of reusing a single
instance avoids that problem:

>>> from contextlib import ExitStack >>> with ExitStack() as outer_stack: … outer_stack.callback(print, “Callback: from outer context”) … with ExitStack() as inner_stack: … inner_stack.callback(print, “Callback: from inner context”) … print(“Leaving inner context”) … print(“Leaving outer context”) … Leaving inner context Callback: from inner context Leaving outer context Callback: from outer context

Context Managers in Python: Using the “with” statement

Context managers are used to set up and tear down temporary contexts, establish and resolve custom settings, and acquire and release resources. The
open()
function for opening files is one of the most familiar examples of a context manager.

Context managers sandwich code blocks between two distinct pieces of logic:

The enter logic – this runs right before the nested code block executes
The exit logic – this runs right after the nested code block is done.

The most common way you’ll work with context managers is by using the
with
statement.

27.Implementing a Context Manager as a Class:¶

At the very least a context manager has an
__enter__
and
__exit__
method defined. Let’s make our own file-opening Context
Manager and learn the basics.

class File(object): def __init__(self, file_name, method): self.file_obj = open(file_name, method) def __enter__(self): return self.file_obj def __exit__(self, type, value, traceback): self.file_obj.close()

Just by defining
__enter__
and
__exit__
methods we can use our new class in
a
with
statement. Let’s try:

with File(‘demo.txt’, ‘w’) as opened_file: opened_file.write(‘Hola!’)

Our
__exit__
method accepts three arguments. They are required by
every
__exit__
method which is a part of a Context Manager class.
Let’s talk about what happens under-the-hood.

The
with
statement stores the
__exit__
method of the
File
class.
It calls the
__enter__
method of the
File
class.
The
__enter__
method opens the file and returns it.
The opened file handle is passed to
opened_file
.
We write to the file using
.write()
.
The
with
statement calls the stored
__exit__
method.
The
__exit__
method closes the file.

LẬP TRÌNH PYTHON CƠ BẢN #24: CONTEXT MANAGER LÀ GÌ ?

Python context manager applications

As you see from the previous example, the common usage of a context manager is to open and close files automatically.

However, you can use context managers in many other cases:

1) Open – Close

If you want to open and close a resource automatically, you can use a context manager.

For example, you can open a socket and close it using a context manager.

2) Lock – release

Context managers can help you manage locks for objects more effectively. They allow you to acquire a lock and release it automatically.

3) Start – stop

Context managers also help you to work with a scenario that requires the start and stop phases.

For example, you can use a context manager to start a timer and stop it automatically.

3) Change – reset

Context managers can work with change and reset scenario.

For example, your application needs to connect to multiple data sources. And it has a default connection.

To connect to another data source:

First, use a context manager to change the default connection to a new one.
Second, work with the new connection
Third, reset it back to the default connection once you complete working with the new connection.

Coding Class-Based Context Managers

To implement the context management protocol and create class-based context managers, you need to add both the
.__enter__()
and the
__exit__()
special methods to your classes. The table below summarizes how these methods work, the arguments they take, and the logic you can put in them:

Method	Description
This method handles the setup logic and is called when entering a new
This method handles the teardown logic and is called when the flow of execution leaves the

When the
with
statement executes, it calls
.__enter__()
on the context manager object to signal that you’re entering into a new runtime context. If you provide a target variable with the
as
specifier, then the return value of
.__enter__()
is assigned to that variable.

When the flow of execution leaves the context,
.__exit__()
is called. If no exception occurs in the
with
code block, then the three last arguments to
.__exit__()
are set to
None
. Otherwise, they hold the type, value, and traceback associated with the exception at hand.

If the
.__exit__()
method returns
True
, then any exception that occurs in the
with
block is swallowed and the execution continues at the next statement after
with
. If
.__exit__()
returns
False
, then exceptions are propagated out of the context. This is also the default behavior when the method doesn’t return anything explicitly. You can take advantage of this feature to encapsulate exception handling inside the context manager.

Writing a Sample Class-Based Context Manager

Here’s a sample class-based context manager that implements both methods,
.__enter__()
and
.__exit__()
. It also shows how Python calls them in a
with
construct:

>>> class HelloContextManager: ... def __enter__(self): ... print("Entering the context...") ... return "Hello, World!" ... def __exit__(self, exc_type, exc_value, exc_tb): ... print("Leaving the context...") ... print(exc_type, exc_value, exc_tb, sep="\n") ... >>> with HelloContextManager() as hello: ... print(hello) ... Entering the context... Hello, World! Leaving the context... None None None

HelloContextManager
implements both
.__enter__()
and
.__exit__()
. In
.__enter__()
, you first print a message to signal that the flow of execution is entering a new context. Then you return the
"Hello, World!"
string. In
.__exit__()
, you print a message to signal that the flow of execution is leaving the context. You also print the content of its three arguments.

When the
with
statement runs, Python creates a new instance of
HelloContextManager
and calls its
.__enter__()
method. You know this because you get
Entering the context...
printed on the screen.

Note: A common mistake when you’re using context managers is forgetting to call the object passed to the
with
statement.

In this case, the statement can’t get the required context manager, and you get an
AttributeError
like this:

>>> with HelloContextManager as hello: ... print(hello) ... Traceback (most recent call last): File " ", line 1, in AttributeError: __enter__

The exception message doesn’t say too much, and you might feel confused in this kind of situation. So, make sure to call the object in the
with
statement to provide the corresponding context manager.

Then Python runs the
with
code block, which prints
hello
to the screen. Note that
hello
holds the return value of
.__enter__()
.

When the flow of execution exits the
with
code block, Python calls
.__exit__()
. You know that because you get
Leaving the context...
printed on your screen. The final line in the output confirms that the three arguments to
.__exit__()
are set to
None
.

Note: A common trick when you don’t remember the exact signature of
.__exit__()
and don’t need to access its arguments is to use
*args
and
**kwargs
like in
def __exit__(self, *args, **kwargs):
.

Now, what happens if an exception occurs during the execution of the
with
block? Go ahead and write the following
with
statement:

>>> with HelloContextManager() as hello: ... print(hello) ... hello[100] ... Entering the context... Hello, World! Leaving the context... string index out of range Traceback (most recent call last): File " ", line 3, in IndexError: string index out of range

In this case, you try to retrieve the value at index
100
in the string
"Hello, World!"
. This raises an
IndexError
, and the arguments to
.__exit__()
are set to the following:

exc_type
is the exception class,
IndexError
.
exc_value
is the exception instance.
exc_tb
is the traceback object.

This behavior is quite useful when you want to encapsulate the exception handling in your context managers.

Handling Exceptions in a Context Manager

As an example of encapsulating exception handling in a context manager, say you expect
IndexError
to be the most common exception when you’re working with
HelloContextManager
. You might want to handle that exception in the context manager so you don’t have to repeat the exception-handling code in every
with
code block. In that case, you can do something like this:

# exc_handling.py class HelloContextManager: def __enter__(self): print("Entering the context...") return "Hello, World!" def __exit__(self, exc_type, exc_value, exc_tb): print("Leaving the context...") if isinstance(exc_value, IndexError): # Handle IndexError here... print(f"An exception occurred in your with block: {exc_type}") print(f"Exception message: {exc_value}") return True with HelloContextManager() as hello: print(hello) hello[100] print("Continue normally from here...")

In
.__exit__()
, you check if
exc_value
is an instance of
IndexError
. If so, then you print a couple of informative messages and finally return with
True
. Returning a truthy value makes it possible to swallow the exception and continue the normal execution after the
with
code block.

In this example, if no
IndexError
occurs, then the method returns
None
and the exception propagates out. However, if you want to be more explicit, then you can return
False
from outside the
if
block.

If you run
exc_handling.py
from your command line, then you get the following output:

$ python exc_handling.py Entering the context... Hello, World! Leaving the context... An exception occurred in your with block: Exception message: string index out of range Continue normally from here...

HelloContextManager
is now able to handle
IndexError
exceptions that occur in the
with
code block. Since you return
True
when an
IndexError
occurs, the flow of execution continues in the next line, right after exiting the
with
code block.

Opening Files for Writing: First Version

Now that you know how to implement the context management protocol, you can get a sense of what this would look like by coding a practical example. Here’s how you can take advantage of
open()
to create a context manager that opens files for writing:

# writable.py class WritableFile: def __init__(self, file_path): self.file_path = file_path def __enter__(self): self.file_obj = open(self.file_path, mode="w") return self.file_obj def __exit__(self, exc_type, exc_val, exc_tb): if self.file_obj: self.file_obj.close()

WritableFile
implements the context management protocol and supports the
with
statement, just like the original
open()
does, but it always opens the file for writing using the
"w"
mode. Here’s how you can use your new context manager:

>>> from writable import WritableFile >>> with WritableFile("hello.txt") as file: ... file.write("Hello, World!") ...

After running this code, your
hello.txt
file contains the
"Hello, World!"
string. As an exercise, you can write a complementary context manager that opens files for reading, but using
pathlib
functionalities. Go ahead and give it a shot!

Redirecting the Standard Output

A subtle detail to consider when you’re writing your own context managers is that sometimes you don’t have a useful object to return from
.__enter__()
and therefore to assign to the
with
target variable. In those cases, you can return
None
explicitly or you can just rely on Python’s implicit return value, which is
None
as well.

For example, say you need to temporarily redirect the standard output,
sys.stdout
, to a given file on your disk. To do this, you can create a context manager like this:

# redirect.py import sys class RedirectedStdout: def __init__(self, new_output): self.new_output = new_output def __enter__(self): self.saved_output = sys.stdout sys.stdout = self.new_output def __exit__(self, exc_type, exc_val, exc_tb): sys.stdout = self.saved_output

This context manager takes a file object through its constructor. In
.__enter__()
, you reassign the standard output,
sys.stdout
, to an instance attribute to avoid losing the reference to it. Then you reassign the standard output to point to the file on your disk. In
.__exit__()
, you just restore the standard output to its original value.

To use
RedirectedStdout
, you can do something like this:

>>> from redirect import RedirectedStdout >>> with open("hello.txt", "w") as file: ... with RedirectedStdout(file): ... print("Hello, World!") ... print("Back to the standard output...") ... Back to the standard output...

The outer
with
statement in this example provides the file object that you’re going to use as your new output,
hello.txt
. The inner
with
temporarily redirects the standard output to
hello.txt
, so the first call to
print()
writes directly to that file instead of printing
"Hello, World!"
on your screen. Note that when you leave the inner
with
code block, the standard output goes back to its original value.

RedirectedStdout
is a quick example of a context manager that doesn’t have a useful value to return from
.__enter__()
. However, if you’re only redirecting the
print()
output, you can get the same functionality without the need for coding a context manager. You just need to provide a
file
argument to
print()
like this:

>>> with open("hello.txt", "w") as file: ... print("Hello, World!", file=file) ...

In this examples,
print()
takes your
hello.txt
file as an argument. This causes
print()
to write directly into the physical file on your disk instead of printing
"Hello, World!"
to your screen.

Measuring Execution Time

Just like every other class, a context manager can encapsulate some internal state. The following example shows how to create a stateful context manager to measure the execution time of a given code block or function:

# timing.py from time import perf_counter class Timer: def __enter__(self): self.start = perf_counter() self.end = 0.0 return lambda: self.end - self.start def __exit__(self, *args): self.end = perf_counter()

When you use
Timer
in a
with
statement,
.__enter__()
gets called. This method uses
time.perf_counter()
to get the time at the beginning of the
with
code block and stores it in
.start
. It also initializes
.end
and returns a
lambda
function that computes a time delta. In this case,
.start
holds the initial state or time measurement.

Note: To take a deeper dive into how to time your code, check out Python Timer Functions: Three Ways to Monitor Your Code.

Once the
with
block ends,
.__exit__()
gets called. The method gets the time at the end of the block and updates the value of
.end
so that the
lambda
function can compute the time required to run the
with
code block.

Here’s how you can use this context manager in your code:

>>> from time import sleep >>> from timing import Timer >>> with Timer() as timer: ... # Time-consuming code goes here... ... sleep(0.5) ... >>> timer() 0.5005456680000862

With
Timer
, you can measure the execution time of any piece of code. In this example,
timer
holds an instance of the
lambda
function that computes the time delta, so you need to call
timer()
to get the final result.

i loved you , i might be able to talk about it more / romantic playlist

Python3

class
FileManager():

def
__init__(
self
, filename, mode):

self
.filename
filename

self
.mode
mode

self
file
None

def
__enter__(
self
):

self
file
open
self
.filename,
self
.mode)

return
self
file

def
__exit__(
self
, exc_type, exc_value, exc_traceback):

self
file
.close()

with FileManager(
'test.txt'
'w'
) as f:

f.write(
'Test'

print
(f.closed)

Output:

True

File management using context manager and with statement: On executing the with block, the following operations happen in sequence:

A FileManager object is created with test.txt as the filename and w(write) as the mode when __init__ method is executed.
The __enter__ method opens the test.txt file in write mode(setup operation) and returns a file object to variable f.
The text ‘Test’ is written into the file.
The __exit__ method takes care of closing the file on exiting the with block(teardown operation). When print(f.closed) is run, the output is True as the FileManager has already taken care of closing the file which otherwise needed to be explicitly done.

Database connection management using context manager: Let’s create a simple database connection management system. The number of database connections that can be opened at a time is also limited(just like file descriptors). Therefore context managers are helpful in managing connections to the database as there could be chances that the programmer may forget to close the connection.

Conclusion

Context managers in Python is one of those topics that a lot of programmers have used but do not understand clearly.

I hope this article has cleared up some of your confusions.

If you’d like to connect to me, I am always available on LinkedIn. Feel free to shoot a message and I’d be happy to respond. Also, if you think this was helpful, consider endorsing my relevant skills on the platform.

Until the next one, take care and keep exploring.

contextlib — Utilities for with-statement contexts¶

Source code: Lib/contextlib.py

This module provides utilities for common tasks involving the
with
statement. For more information see also Context Manager Types and
With Statement Context Managers.

من دولار ل مليون!!.. كيف تحمي أموالك من التضخم وتستثمرها بنجاح؟

Implementing Python context manager protocol

The following shows a simple implementation of the
open()
function using the context manager protocol:

class File: def __init__(self, filename, mode): self.filename = filename self.mode = mode def __enter__(self): print(f'Opening the file {self.filename}.') self.__file = open(self.filename, self.mode) return self.__file def __exit__(self, exc_type, exc_value, exc_traceback): print(f'Closing the file {self.filename}.') if not self.__file.closed: self.__file.close() return False with File('data.txt', 'r') as f: print(int(next(f)))
Code language: Python (python)

How it works.

First, initialize the
filename
and
mode
in the
__init__()
method.
Second, open the file in the
__enter__()
method and return the file object.
Third, close the file if it’s open in the
__exit__()
method.

Utilities¶

Functions and classes provided:

class contextlib.AbstractContextManager¶

An abstract base class for classes that implement

object.__enter__()
and
object.__exit__()
. A default implementation for
object.__enter__()
is provided which returns
self
while
object.__exit__()
is an abstract method which by default returns
None
. See also the definition of Context Manager Types.

New in version 3.6.

class contextlib.AbstractAsyncContextManager¶

An abstract base class for classes that implement

object.__aenter__()
and
object.__aexit__()
. A default implementation for
object.__aenter__()
is provided which returns
self
while
object.__aexit__()
is an abstract method which by default returns
None
. See also the definition of Asynchronous Context Managers.

New in version 3.7.

@contextlib.contextmanager¶

This function is a decorator that can be used to define a factory function for

with
statement context managers, without needing to create a class or separate
__enter__()
and
__exit__()
methods.

While many objects natively support use in with statements, sometimes a resource needs to be managed that isn’t a context manager in its own right, and doesn’t implement a

close()
method for use with
contextlib.closing

An abstract example would be the following to ensure correct resource management:

from contextlib import contextmanager @contextmanager def managed_resource(*args, **kwds): # Code to acquire resource, e.g.: resource = acquire_resource(*args, **kwds) try: yield resource finally: # Code to release resource, e.g.: release_resource(resource)

The function can then be used like this:

>>> with managed_resource(timeout=3600) as resource: … # Resource is released at the end of this block, … # even if code in the block raises an exception

The function being decorated must return a generator-iterator when called. This iterator must yield exactly one value, which will be bound to the targets in the

with
statement’s
as
clause, if any.

At the point where the generator yields, the block nested in the

with
statement is executed. The generator is then resumed after the block is exited. If an unhandled exception occurs in the block, it is reraised inside the generator at the point where the yield occurred. Thus, you can use a
try
…
except
…
finally
statement to trap the error (if any), or ensure that some cleanup takes place. If an exception is trapped merely in order to log it or to perform some action (rather than to suppress it entirely), the generator must reraise that exception. Otherwise the generator context manager will indicate to the
with
statement that the exception has been handled, and execution will resume with the statement immediately following the
with
statement.
contextmanager()
uses
ContextDecorator
so the context managers it creates can be used as decorators as well as in
with
statements. When used as a decorator, a new generator instance is implicitly created on each function call (this allows the otherwise “one-shot” context managers created by
contextmanager()
to meet the requirement that context managers support multiple invocations in order to be used as decorators).

Changed in version 3.2: Use of

ContextDecorator
.

@contextlib.asynccontextmanager¶

Similar to

contextmanager()
, but creates an asynchronous context manager.

This function is a decorator that can be used to define a factory function for

async with
statement asynchronous context managers, without needing to create a class or separate
__aenter__()
and
__aexit__()
methods. It must be applied to an asynchronous generator function.

A simple example:

from contextlib import asynccontextmanager @asynccontextmanager async def get_connection(): conn = await acquire_db_connection() try: yield conn finally: await release_db_connection(conn) async def get_all_users(): async with get_connection() as conn: return conn.query(‘SELECT …’)

New in version 3.7.

Context managers defined with

asynccontextmanager()
can be used either as decorators or with
async with
statements:

import time from contextlib import asynccontextmanager @asynccontextmanager async def timeit(): now = time.monotonic() try: yield finally: print(f’it took {time.monotonic() – now}s to run’) @timeit() async def main(): # … async code …

When used as a decorator, a new generator instance is implicitly created on each function call. This allows the otherwise “one-shot” context managers created by

asynccontextmanager()
to meet the requirement that context managers support multiple invocations in order to be used as decorators.

Changed in version 3.10: Async context managers created with

asynccontextmanager()
can be used as decorators.

contextlib.closing(thing)¶

Return a context manager that closes thing upon completion of the block. This is basically equivalent to:

from contextlib import contextmanager @contextmanager def closing(thing): try: yield thing finally: thing.close()

And lets you write code like this:

from contextlib import closing from urllib.request import urlopen with closing(urlopen(‘https://www.python.org’)) as page: for line in page: print(line)

without needing to explicitly close

page
. Even if an error occurs,
page.close()
will be called when the
with
block is exited.

Note

Most types managing resources support the context manager protocol, which closes thing on leaving the

with
statement. As such,
closing()
is most useful for third party types that don’t support context managers. This example is purely for illustration purposes, as
urlopen()
would normally be used in a context manager.

contextlib.aclosing(thing)¶

Return an async context manager that calls the

aclose()
method of thing upon completion of the block. This is basically equivalent to:

from contextlib import asynccontextmanager @asynccontextmanager async def aclosing(thing): try: yield thing finally: await thing.aclose()

Significantly,

aclosing()
supports deterministic cleanup of async generators when they happen to exit early by
break
or an exception. For example:

from contextlib import aclosing async with aclosing(my_generator()) as values: async for value in values: if value == 42: break

This pattern ensures that the generator’s async exit code is executed in the same context as its iterations (so that exceptions and context variables work as expected, and the exit code isn’t run after the lifetime of some task it depends on).

New in version 3.10.

contextlib.nullcontext(enter_result=None)¶

Return a context manager that returns enter_result from

__enter__
, but otherwise does nothing. It is intended to be used as a stand-in for an optional context manager, for example:

def myfunction(arg, ignore_exceptions=False): if ignore_exceptions: # Use suppress to ignore all exceptions. cm = contextlib.suppress(Exception) else: # Do not ignore any exceptions, cm has no effect. cm = contextlib.nullcontext() with cm: # Do something

An example using enter_result:

def process_file(file_or_path): if isinstance(file_or_path, str): # If string, open file cm = open(file_or_path) else: # Caller is responsible for closing file cm = nullcontext(file_or_path) with cm as file: # Perform processing on the file

It can also be used as a stand-in for asynchronous context managers:

async def send_http(session=None): if not session: # If no http session, create it with aiohttp cm = aiohttp.ClientSession() else: # Caller is responsible for closing the session cm = nullcontext(session) async with cm as session: # Send http requests with session

New in version 3.7.

Changed in version 3.10: asynchronous context manager support was added.

contextlib.suppress(*exceptions)¶

Return a context manager that suppresses any of the specified exceptions if they occur in the body of a

with
statement and then resumes execution with the first statement following the end of the
with
statement.

As with any other mechanism that completely suppresses exceptions, this context manager should be used only to cover very specific errors where silently continuing with program execution is known to be the right thing to do.

For example:

from contextlib import suppress with suppress(FileNotFoundError): os.remove(‘somefile.tmp’) with suppress(FileNotFoundError): os.remove(‘someotherfile.tmp’)

This code is equivalent to:

try: os.remove(‘somefile.tmp’) except FileNotFoundError: pass try: os.remove(‘someotherfile.tmp’) except FileNotFoundError: pass

This context manager is reentrant.

If the code within the

with
block raises a
BaseExceptionGroup
, suppressed exceptions are removed from the group. If any exceptions in the group are not suppressed, a group containing them is re-raised.

New in version 3.4.

Changed in version 3.12:

suppress
now supports suppressing exceptions raised as part of an
BaseExceptionGroup
.

contextlib.redirect_stdout(new_target)¶

Context manager for temporarily redirecting

sys.stdout
to another file or file-like object.

This tool adds flexibility to existing functions or classes whose output is hardwired to stdout.

For example, the output of

help()
normally is sent to sys.stdout. You can capture that output in a string by redirecting the output to an
io.StringIO
object. The replacement stream is returned from the
__enter__
method and so is available as the target of the
with
statement:

with redirect_stdout(io.StringIO()) as f: help(pow) s = f.getvalue()

To send the output of

help()
to a file on disk, redirect the output to a regular file:

with open(‘help.txt’, ‘w’) as f: with redirect_stdout(f): help(pow)

To send the output of

help()
to sys.stderr:

with redirect_stdout(sys.stderr): help(pow)

Note that the global side effect on

sys.stdout
means that this context manager is not suitable for use in library code and most threaded applications. It also has no effect on the output of subprocesses. However, it is still a useful approach for many utility scripts.

This context manager is reentrant.

New in version 3.4.

contextlib.redirect_stderr(new_target)¶

Similar to

redirect_stdout()
but redirecting
sys.stderr
to another file or file-like object.

This context manager is reentrant.

New in version 3.5.

contextlib.chdir(path)¶

Non parallel-safe context manager to change the current working directory. As this changes a global state, the working directory, it is not suitable for use in most threaded or async contexts. It is also not suitable for most non-linear code execution, like generators, where the program execution is temporarily relinquished – unless explicitly desired, you should not yield when this context manager is active.

This is a simple wrapper around

chdir()
, it changes the current working directory upon entering and restores the old one on exit.

This context manager is reentrant.

New in version 3.11.

class contextlib.ContextDecorator¶

A base class that enables a context manager to also be used as a decorator.

Context managers inheriting from

ContextDecorator
have to implement
__enter__
and
__exit__
as normal.
__exit__
retains its optional exception handling even when used as a decorator.
ContextDecorator
is used by
contextmanager()
, so you get this functionality automatically.

Example of

ContextDecorator
:

from contextlib import ContextDecorator class mycontext(ContextDecorator): def __enter__(self): print(‘Starting’) return self def __exit__(self, *exc): print(‘Finishing’) return False

The class can then be used like this:

>>> @mycontext() … def function(): … print(‘The bit in the middle’) … >>> function() Starting The bit in the middle Finishing >>> with mycontext(): … print(‘The bit in the middle’) … Starting The bit in the middle Finishing

This change is just syntactic sugar for any construct of the following form:

def f(): with cm(): # Do stuff

ContextDecorator
lets you instead write:

@cm() def f(): # Do stuff

It makes it clear that the

cm
applies to the whole function, rather than just a piece of it (and saving an indentation level is nice, too).

Existing context managers that already have a base class can be extended by using

ContextDecorator
as a mixin class:

from contextlib import ContextDecorator class mycontext(ContextBaseClass, ContextDecorator): def __enter__(self): return self def __exit__(self, *exc): return False

Note

As the decorated function must be able to be called multiple times, the underlying context manager must support use in multiple

with
statements. If this is not the case, then the original construct with the explicit
with
statement inside the function should be used.

New in version 3.2.

class contextlib.AsyncContextDecorator¶

Similar to

ContextDecorator
but only for asynchronous functions.

Example of

AsyncContextDecorator
:

from asyncio import run from contextlib import AsyncContextDecorator class mycontext(AsyncContextDecorator): async def __aenter__(self): print(‘Starting’) return self async def __aexit__(self, *exc): print(‘Finishing’) return False

The class can then be used like this:

>>> @mycontext() … async def function(): … print(‘The bit in the middle’) … >>> run(function()) Starting The bit in the middle Finishing >>> async def function(): … async with mycontext(): … print(‘The bit in the middle’) … >>> run(function()) Starting The bit in the middle Finishing

New in version 3.10.

class contextlib.ExitStack¶

A context manager that is designed to make it easy to programmatically combine other context managers and cleanup functions, especially those that are optional or otherwise driven by input data.

For example, a set of files may easily be handled in a single with statement as follows:

with ExitStack() as stack: files = [stack.enter_context(open(fname)) for fname in filenames] # All opened files will automatically be closed at the end of # the with statement, even if attempts to open files later # in the list raise an exception

The

__enter__()
method returns the
ExitStack
instance, and performs no additional operations.

Each instance maintains a stack of registered callbacks that are called in reverse order when the instance is closed (either explicitly or implicitly at the end of a

with
statement). Note that callbacks are not invoked implicitly when the context stack instance is garbage collected.

This stack model is used so that context managers that acquire their resources in their

__init__
method (such as file objects) can be handled correctly.

Since registered callbacks are invoked in the reverse order of registration, this ends up behaving as if multiple nested

with
statements had been used with the registered set of callbacks. This even extends to exception handling – if an inner callback suppresses or replaces an exception, then outer callbacks will be passed arguments based on that updated state.

This is a relatively low level API that takes care of the details of correctly unwinding the stack of exit callbacks. It provides a suitable foundation for higher level context managers that manipulate the exit stack in application specific ways.

New in version 3.3.

enter_context(cm)¶

Enters a new context manager and adds its

__exit__()
method to the callback stack. The return value is the result of the context manager’s own
__enter__()
method.

These context managers may suppress exceptions just as they normally would if used directly as part of a

with
statement.

Changed in version 3.11: Raises

TypeError
instead of
AttributeError
if cm is not a context manager.

push(exit)¶

Adds a context manager’s

__exit__()
method to the callback stack.

__enter__
is not invoked, this method can be used to cover part of an
__enter__()
implementation with a context manager’s own
__exit__()
method.

If passed an object that is not a context manager, this method assumes it is a callback with the same signature as a context manager’s

__exit__()
method and adds it directly to the callback stack.

By returning true values, these callbacks can suppress exceptions the same way context manager

__exit__()
methods can.

The passed in object is returned from the function, allowing this method to be used as a function decorator.

callback(callback, /, *args, **kwds)¶

Accepts an arbitrary callback function and arguments and adds it to the callback stack.

Unlike the other methods, callbacks added this way cannot suppress exceptions (as they are never passed the exception details).

The passed in callback is returned from the function, allowing this method to be used as a function decorator.

pop_all()¶

Transfers the callback stack to a fresh

ExitStack
instance and returns it. No callbacks are invoked by this operation – instead, they will now be invoked when the new stack is closed (either explicitly or implicitly at the end of a
with
statement).

For example, a group of files can be opened as an “all or nothing” operation as follows:

with ExitStack() as stack: files = [stack.enter_context(open(fname)) for fname in filenames] # Hold onto the close method, but don’t call it yet. close_files = stack.pop_all().close # If opening any file fails, all previously opened files will be # closed automatically. If all files are opened successfully, # they will remain open even after the with statement ends. # close_files() can then be invoked explicitly to close them all.

close()¶: Immediately unwinds the callback stack, invoking callbacks in the reverse order of registration. For any context managers and exit callbacks registered, the arguments passed in will indicate that no exception occurred.

class contextlib.AsyncExitStack¶

An asynchronous context manager, similar to

ExitStack
, that supports combining both synchronous and asynchronous context managers, as well as having coroutines for cleanup logic.

The

close()
method is not implemented;
aclose()
must be used instead.

coroutine enter_async_context(cm)¶

Similar to

ExitStack.enter_context()
but expects an asynchronous context manager.

Changed in version 3.11: Raises

TypeError
instead of
AttributeError
if cm is not an asynchronous context manager.

push_async_exit(exit)¶

Similar to

ExitStack.push()
but expects either an asynchronous context manager or a coroutine function.

push_async_callback(callback, /, *args, **kwds)¶

Similar to

ExitStack.callback()
but expects a coroutine function.

coroutine aclose()¶

Similar to

ExitStack.close()
but properly handles awaitables.

Continuing the example for

asynccontextmanager()
:

async with AsyncExitStack() as stack: connections = [await stack.enter_async_context(get_connection()) for i in range(5)] # All opened connections will automatically be released at the end of # the async with statement, even if attempts to open a connection # later in the list raise an exception.

New in version 3.7.

Introduction to Python context managers

A context manager is an object that defines a runtime context executing within the
with
statement.

Let’s start with a simple example to understand the context manager concept.

Suppose that you have a file called
data.txt
that contains an integer
100
.

The following program reads the
data.txt
file, converts its contents to a number, and shows the result to the standard output:

f = open('data.txt') data = f.readlines() # convert the number to integer and display it print(int(data[0])) f.close()
Code language: Python (python)

The code is simple and straightforward.

However, the
data.txt
may contain data that cannot be converted to a number. In this case, the code will result in an exception.

For example, if the
data.txt
contains the string
'100'
instead of the number 100, you’ll get the following error:

ValueError: invalid literal for int() with base 10: "'100'"
Code language: Python (python)

Because of this exception, Python may not close the file properly.

To fix this, you may use the
try...except...finally
statement:

try: f = open('data.txt') data = f.readlines() # convert the number to integer and display it print(int(data[0])) except ValueError as error: print(error) finally: f.close()
Code language: Python (python)

Since the code in the
finally
block always executes, the code will always close the file properly.

This solution works as expected. However, it’s quite verbose.

Therefore, Python provides you with a better way that allows you to automatically close the file after you complete processing it.

This is where context managers come into play.

The following shows how to use a context manager to process the
data.txt
file:

with open('data.txt') as f: data = f.readlines() print(int(data[0])
Code language: Python (python)

In this example, we use the
open()
function with the
with
statement. After the
with
block, Python will close automatically.

Creating an Asynchronous Context Manager

To create an asynchronous context manager, you need to define the
.__aenter__()
and
.__aexit__()
methods. The script below is a reimplementation of the original script
site_checker_v0.py
you saw before, but this time you provide a custom asynchronous context manager to wrap the session creation and closing functionalities:

# site_checker_v1.py import aiohttp import asyncio class AsyncSession: def __init__(self, url): self._url = url async def __aenter__(self): self.session = aiohttp.ClientSession() response = await self.session.get(self._url) return response async def __aexit__(self, exc_type, exc_value, exc_tb): await self.session.close() async def check(url): async with AsyncSession(url) as response: print(f"{url}: status -> {response.status}") html = await response.text() print(f"{url}: type -> {html[:17].strip()}") async def main(): await asyncio.gather( check("https://realpython.com"), check("https://pycoders.com"), ) asyncio.run(main())

This script works similar to its previous version,
site_checker_v0.py
. The main difference is that, in this example, you extract the logic of the original outer
async with
statement and encapsulate it in
AsyncSession
.

In
.__aenter__()
, you create an
aiohttp.ClientSession()
, await the
.get()
response, and finally return the response itself. In
.__aexit__()
, you close the session, which corresponds to the teardown logic in this specific case. Note that
.__aenter__()
and
.__aexit__()
must return awaitable objects. In other words, you must define them with
async def
, which returns a coroutine object that is awaitable by definition.

If you run the script from your command line, then you get an output similar to this:

$ python site_checker_v1.py https://realpython.com: status -> 200 https://pycoders.com: status -> 200 https://realpython.com: type ->
https://pycoders.com: type ->

Great! Your script works just like its first version. It sends
GET
requests to both sites concurrently and processes the corresponding responses.

Finally, a common practice when you’re writing asynchronous context managers is to implement the four special methods:

.__aenter__()
.__aexit__()
.__enter__()
.__exit__()

This makes your context manager usable with both variations of
with
.

Python API Development - Comprehensive Course for Beginners — Python API Development – Comprehensive Course for Beginners

Python with statement

Here is the typical syntax of the
with
statement:

with context as ctx: # use the the object # context is cleaned up
Code language: Python (python)

How it works.

When Python encounters the
with
statement, it creates a new context. The context can optionally return an
object
.
After the
with
block, Python cleans up the context automatically.
The scope of the
ctx
has the same scope as the
with
statement. It means that you can access the
ctx
both inside and after the
with
statement.

The following shows how to access the variable after the
with
statement:

with open('data.txt') as f: data = f.readlines() print(int(data[0])) print(f.closed) # True
Code language: Python (python)

The with statement

A
with
statement is the primary method used to call a context manager. The general syntax is as follows:

This code works in the following order:

SomeContextManager
executes its enter (setup) logic before the indented code runs.
SomeContextManager
binds a value to
context_variable
, which can be used in the indented code
Inner code block runs:
# do stuff
.
SomeContextManager
executes its exit (cleanup) logic.

So why is this useful? In the next section, we’ll discuss why this programming pattern comes in handy and why it’s worth making your context managers from time to time.

Build a Comment Toxicity Model with Deep Learning and Python

Examples and Recipes¶

This section describes some examples and recipes for making effective use of
the tools provided by
contextlib
.

Supporting a variable number of context managers¶

The primary use case for
ExitStack
is the one given in the class
documentation: supporting a variable number of context managers and other
cleanup operations in a single
with
statement. The variability
may come from the number of context managers needed being driven by user
input (such as opening a user specified collection of files), or from
some of the context managers being optional:

with ExitStack() as stack: for resource in resources: stack.enter_context(resource) if need_special_resource(): special = acquire_special_resource() stack.callback(release_special_resource, special) # Perform operations that use the acquired resources

As shown,
ExitStack
also makes it quite easy to use
with
statements to manage arbitrary resources that don’t natively support the
context management protocol.

Catching exceptions from enter methods¶

It is occasionally desirable to catch exceptions from an
__enter__
method implementation, without inadvertently catching exceptions from
the
with
statement body or the context manager’s
__exit__
method. By using
ExitStack
the steps in the context management
protocol can be separated slightly in order to allow this:

stack = ExitStack() try: x = stack.enter_context(cm) except Exception: # handle __enter__ exception else: with stack: # Handle normal case

Actually needing to do this is likely to indicate that the underlying API
should be providing a direct resource management interface for use with
try
/
except
/
finally
statements, but not
all APIs are well designed in that regard. When a context manager is the
only resource management API provided, then
ExitStack
can make it
easier to handle various situations that can’t be handled directly in a
with
statement.

Cleaning up in an enter implementation¶

As noted in the documentation of
ExitStack.push()
, this
method can be useful in cleaning up an already allocated resource if later
steps in the
__enter__()
implementation fail.

Here’s an example of doing this for a context manager that accepts resource acquisition and release functions, along with an optional validation function, and maps them to the context management protocol:

from contextlib import contextmanager, AbstractContextManager, ExitStack class ResourceManager(AbstractContextManager): def __init__(self, acquire_resource, release_resource, check_resource_ok=None): self.acquire_resource = acquire_resource self.release_resource = release_resource if check_resource_ok is None: def check_resource_ok(resource): return True self.check_resource_ok = check_resource_ok @contextmanager def _cleanup_on_error(self): with ExitStack() as stack: stack.push(self) yield # The validation check passed and didn’t raise an exception # Accordingly, we want to keep the resource, and pass it # back to our caller stack.pop_all() def __enter__(self): resource = self.acquire_resource() with self._cleanup_on_error(): if not self.check_resource_ok(resource): msg = “Failed validation for {!r}” raise RuntimeError(msg.format(resource)) return resource def __exit__(self, *exc_details): # We don’t need to duplicate any of our resource release logic self.release_resource()

Replacing any use of try-finally and flag variables¶

A pattern you will sometimes see is a
try-finally
statement with a flag
variable to indicate whether or not the body of the
finally
clause should
be executed. In its simplest form (that can’t already be handled just by
using an
except
clause instead), it looks something like this:

cleanup_needed = True try: result = perform_operation() if result: cleanup_needed = False finally: if cleanup_needed: cleanup_resources()

As with any
try
statement based code, this can cause problems for
development and review, because the setup code and the cleanup code can end
up being separated by arbitrarily long sections of code.

ExitStack
makes it possible to instead register a callback for
execution at the end of a
with
statement, and then later decide to skip
executing that callback:

from contextlib import ExitStack with ExitStack() as stack: stack.callback(cleanup_resources) result = perform_operation() if result: stack.pop_all()

This allows the intended cleanup up behaviour to be made explicit up front, rather than requiring a separate flag variable.

If a particular application uses this pattern a lot, it can be simplified even further by means of a small helper class:

from contextlib import ExitStack class Callback(ExitStack): def __init__(self, callback, /, *args, **kwds): super().__init__() self.callback(callback, *args, **kwds) def cancel(self): self.pop_all() with Callback(cleanup_resources) as cb: result = perform_operation() if result: cb.cancel()

If the resource cleanup isn’t already neatly bundled into a standalone
function, then it is still possible to use the decorator form of
ExitStack.callback()
to declare the resource cleanup in
advance:

from contextlib import ExitStack with ExitStack() as stack: @stack.callback def cleanup_resources(): … result = perform_operation() if result: stack.pop_all()

Due to the way the decorator protocol works, a callback function declared this way cannot take any parameters. Instead, any resources to be released must be accessed as closure variables.

Using a context manager as a function decorator¶

ContextDecorator
makes it possible to use a context manager in
both an ordinary
with
statement and also as a function decorator.

For example, it is sometimes useful to wrap functions or groups of statements
with a logger that can track the time of entry and time of exit. Rather than
writing both a function decorator and a context manager for the task,
inheriting from
ContextDecorator
provides both capabilities in a
single definition:

from contextlib import ContextDecorator import logging logging.basicConfig(level=logging.INFO) class track_entry_and_exit(ContextDecorator): def __init__(self, name): self.name = name def __enter__(self): logging.info(‘Entering: %s’, self.name) def __exit__(self, exc_type, exc, exc_tb): logging.info(‘Exiting: %s’, self.name)

Instances of this class can be used as both a context manager:

with track_entry_and_exit(‘widget loader’): print(‘Some time consuming activity goes here’) load_widget()

And also as a function decorator:

@track_entry_and_exit(‘widget loader’) def activity(): print(‘Some time consuming activity goes here’) load_widget()

Note that there is one additional limitation when using context managers
as function decorators: there’s no way to access the return value of
__enter__()
. If that value is needed, then it is still necessary to use
an explicit
with
statement.

Creating Context Managers

Function-Based Implementation

The standard library provides
contextlib
, a module containing
with
statement utilities. One important helper is the
@contextmanager
decorator, which lets us define function-based context managers.

Let’s try it out by making a context manager function that swaps the case of a string:

The
@contextmanager
decorator expects a
yield
statement in the function body.
yield
divides the function into three parts:

The expressions above
yield
are executed right before the code that is managed.
The managed code runs at
yield
. Whatever is yielded makes up the context variable—
swapped_string
in our case.
The expressions below
yield
are executed after the managed code is run.

Notice that we have placed
yield
within a
try-except-finally
block. This is not enforced, yet, it is good practice. This way, if an error occurs in the managed code the context manager will carry out its exit logic no matter what.

Look what happens if we raise a
ValueError
inside the
with
statement:

The
except
block within
example_cm
handled the exception, and the
finally
block ensured we saw the exit message.

Notice that the “End of context manager” string is still printed. We’re specifically catching a
ValueError
, but let’s see what happens when an unhandled error is raised:

In this case, the
finally
block still executed regardless of the uncaught error, but the “End of context manager” printout didn’t make it. This exemplifies why a
finally
block is helpful in many situations.

For more flexibility in creating context managers, we’ll now introduce the class-based implementations.

Class-based implementation

Python defines a context management protocol that dictates that any class with an
__enter__()
and an
__exit__()
method can work as a context manager.

Double underscore methods are special methods that are not called but instead triggered. They are internally set to run at specific times or after certain events. For example,
__enter()__
runs when a
with
statement is entered and
__exit__
runs right before the
with
block is left.

We have seen an example of function-based context managers, which work great for quick and simple cases. For more complex use cases, we’ll define context management as an additional ability to an existing class.

Let’s create a
ListProtect
context manager class, which will operate on a copy of a list before returning the changes. This way, the original list would be restored unaltered if an error occurred during the operation.

Let’s see how that looks.

And we can use it like so:

The list was successfully modified. But what if an error occurs?

In this instance,
ListProtect
caught an error and the list remained unaltered. So how does this work?

Explanation

In the
ListProtect
class, we defined two required methods:

__enter__()
– this method defines what happens before the logic under the
with
statement runs. If the enter method returns anything, it is bound to the context variable. In our class, we create a clone of the list and returned it as the context variable—
the_copy
in the above examples.

__exit__()
– this method defines what happens when the
with
logic is complete or has raised an error. Besides
self
, this method takes three parameters:
exc_type
,
exc_val
,
exc_tb
, which can also be shortened to
*exc
. If no exception occurs in the managee, all these values are None. If we return a truthy value at the end of
__exit()__
—as we did in our class—the error is suppressed. Otherwise, if we return a falsy value, such as
None
,
False
, or blank, the error is propagated.

In short,
ListProtect
‘s
__exit()__
method first checks whether the exception type was
None
. If so, there were no errors, and it applies the changes to the original list; otherwise, it announces an error occurred.

Senior Programmers vs Junior Developers #shorts

Python3

from
pymongo
import
MongoClient

class
MongoDBConnectionManager():

def
__init__(
self
, hostname, port):

self
.hostname
hostname

self
.port
port

self
.connection
None

def
__enter__(
self
):

self
.connection
MongoClient(
self
.hostname,
self
.port)

return
self
.connection

def
__exit__(
self
, exc_type, exc_value, exc_traceback):

self
.connection.close()

with MongoDBConnectionManager(
'localhost'
'27017'
) as mongo:

collection
mongo.connection.SampleDb.test

data
collection.find({
'_id'
})

print
(data.get(
'name'
))

Database connection management using context manager and with statement: On executing the with block, the following operations happen in sequence:

A MongoDBConnectionManager object is created with localhost as the hostname name and 27017 as the port when the __init__ method is executed.
The __enter__ method opens the MongoDB connection and returns the MongoClient object to variable mongo.
The test collection in the SampleDb database is accessed and the document with _id=1 is retrieved. The name field of the document is printed.
The __exit__ method takes care of closing the connection on exiting the with block(teardown operation).

Don’t miss your chance to ride the wave of the data revolution! Every industry is scaling new heights by tapping into the power of data. Sharpen your skills and become a part of the hottest trend in the 21st century.

Dive into the future of technology – explore the Complete Machine Learning and Data Science Program by GeeksforGeeks and stay ahead of the curve.

Last Updated :
03 Nov, 2022

Like Article

Save Article

Share your thoughts in the comments

Please Login to comment…

One of the most common tasks that you’ll have to perform in your programs is working with external resources. These resources can be files on your computer’s storage or an open connection to third-party service on the internet.

For the sake of simplicity, imagine a program that opens a file, writes something to it, and then closes the file.

One way to implement this program in Python would be like this:

def main(): my_file = open('books.txt', 'w') my_file.write('If Tomorrow Comes by Sidney Sheldon') my_file.close() if __name__ == '__main__': main()

Given that you run this program with the right permissions on your computer, it’ll create a file called
books.txt
and write
If Tomorrow Comes by Sidney Sheldon
in it.

The
open()
function is one of the built-in functions in Python. It can open a file from a given path and return a corresponding file object.

A file object or file-like object, as it’s often called, is a useful way to encapsulate methods like
read()
,
write()
, or
close()
.

The
write()
method can be used write/send bytes-like object to an open stream, like a file.

Whenever you open an external resource, you must close it when its no longer needed, and the
close()
method does just that.

This program is functional, but it has a big flaw. If the program fails to close the file, it will remain open until the program itself closes.

You see, every program that you run on your computer gets a finite amount of memory allocated to it. All the variables you create or external resource you open from a program stay within the memory allocated to it by your computer.

If a program like this one, keeps opening new files without closing the previous ones, the allocated memory will keep shrinking.

At one point the program will inevitably run out of memory and crash ungracefully. This problem is referred to as a memory leak.

One way to prevent this from happening in Python is using a
try...except...finally
statement.

def main(): my_file = open('books.txt', 'w') try: my_file.write('If Tomorrow Comes by Sidney Sheldon') except Exception as e: print(f'writing to file failed: {e}') finally: my_file.close() if __name__ == '__main__': main()

The code inside the
finally
block will run no matter what. So even if the program fails on the right action, it’ll still be executed.

So, this solves the problem but imagine writing these lines of code every time you want to write something to a file.

It’s not very reusable. You will have to repeat yourself a lot and chances of skipping a portion of the
if...except...finally
ladder is also a possibility.

That’s where context managers come in.

Writing Good APIs With Context Managers

Context managers are quite flexible, and if you use the
with
statement creatively, then you can define convenient APIs for your classes, modules, and packages.

For example, what if the resource you wanted to manage is the text indentation level in some kind of report generator application? In that case, you could write code like this:

with Indenter() as indent: indent.print("hi!") with indent: indent.print("hello") with indent: indent.print("bonjour") indent.print("hey")

This almost reads like a domain-specific language (DSL) for indenting text. Also, notice how this code enters and leaves the same context manager multiple times to switch between different indentation levels. Running this code snippet leads to the following output and prints neatly formatted text:

hi! hello bonjour hey

How would you implement a context manager to support this functionality? This could be a great exercise to wrap your head around how context managers work. So, before you check out the implementation below, you might take some time and try to solve this by yourself as a learning exercise.

Ready? Here’s how you might implement this functionality using a context manager class:

class Indenter: def __init__(self): self.level = -1 def __enter__(self): self.level += 1 return self def __exit__(self, exc_type, exc_value, exc_tb): self.level -= 1 def print(self, text): print(" " * self.level + text)

Here,
.__enter__()
increments
.level
by every time the flow of execution enters the context. The method also returns the current instance,
self
. In
.__exit__()
, you decrease
.level
so the printed text steps back one indentation level every time you exit the context.

The key point in this example is that returning
self
from
.__enter__()
allows you to reuse the same context manager across several nested
with
statements. This changes the text indentation level every time you enter and leave a given context.

A good exercise for you at this point would be to write a function-based version of this context manager. Go ahead and give it a try!

Python context manager protocol

Python context managers work based on the context manager protocol.

The context manager protocol has the following methods:

__enter__()
– setup the context and optionally return some object
__exit__()
– cleanup the object.

If you want a class to support the context manager protocol, you need to implement these two methods.

Suppose that
ContextManager
is a class that supports the context manager protocol.

The following shows how to use the
ContextManager
class:

with ContextManager() as ctx: # do something # done with the context
Code language: Python (python)

When you use
ContextManager
class with the
with
statement, Python implicitly creates an instance of the
ContextManager
class (
instance
) and automatically call
__enter__()
method on that instance.

The
__enter__()
method may optionally return an object. If so, Python assigns the returned object the
ctx
.

Notice that
ctx
references the object returned by the
__enter__()
method. It doesn’t reference the instance of the
ContextManager
class.

If an exception occurs inside the with block or after the
with
block, Python calls the
__exit__()
method on the
instance
object.

Functionally, the
with
statement is equivalent to the following
try...finally
statement:

instance = ContextManager() ctx = instance.__enter__() try: # do something with the txt finally: # done with the context instance.__exit__()
Code language: Python (python)

The enter() method

In the
__enter__()
method, you can carry the necessary steps to setup the context.

Optionally, you can returns an object from the
__enter__()
method.

The exit() method

Python always executes the
__exit__()
method even if an exception occurs in the
with
block.

The
__exit__()
method accepts three arguments: exception type, exception value, and traceback object. All of these arguments will be
None
if no exception occurs.

def __exit__(self, ex_type, ex_value, ex_traceback): ...
Code language: Python (python)

The
__exit__()
method returns a boolean value, either
True
or
False
.

If the return value is True, Python will make any exception silent. Otherwise, it doesn’t silence the exception.

Python3

file_descriptors
[]

for
in
range
100000
):

file_descriptors.append(
open
'test.txt'
'w'
))

Output:

Traceback (most recent call last):
File “context.py”, line 3, in
OSError: [Errno 24] Too many open files: ‘test.txt’

An error message saying that too many files are open. The above example is a case of file descriptor leakage. It happens because there are too many open files and they are not closed. There might be chances where a programmer may forget to close an opened file.

Managing Resources using context manager: Suppose a block of code raises an exception or if it has a complex algorithm with multiple return paths, it becomes cumbersome to close a file in all the places. Generally in other languages when working with files try-except-finally is used to ensure that the file resource is closed after usage even if there is an exception. Python provides an easy way to manage resources: Context Managers. The with keyword is used. When it gets evaluated it should result in an object that performs context management. Context managers can be written using classes or functions(with decorators).

Creating a Context Manager: When creating context managers using classes, user need to ensure that the class has the methods: __enter__() and __exit__(). The __enter__() returns the resource that needs to be managed and the __exit__() does not return anything but performs the cleanup operations. First, let us create a simple class called ContextManager to understand the basic structure of creating context managers using classes, as shown below:

Context API là gì? Cách sử dụng useContext() trong React

Creating Function-Based Context Managers

Python’s generator functions and the
contextlib.contextmanager
decorator provide an alternative and convenient way to implement the context management protocol. If you decorate an appropriately coded generator function with
@contextmanager
, then you get a function-based context manager that automatically provides both required methods,
.__enter__()
and
.__exit__()
. This can make your life more pleasant by saving you some boilerplate code.

The general pattern to create a context manager using
@contextmanager
along with a generator function goes like this:

>>> from contextlib import contextmanager >>> @contextmanager ... def hello_context_manager(): ... print("Entering the context...") ... yield "Hello, World!" ... print("Leaving the context...") ... >>> with hello_context_manager() as hello: ... print(hello) ... Entering the context... Hello, World! Leaving the context...

In this example, you can identify two visible sections in
hello_context_manager()
. Before the
yield
statement, you have the setup section. There, you can place the code that acquires the managed resources. Everything before the
yield
runs when the flow of execution enters the context.

After the
yield
statement, you have the teardown section, in which you can release the resources and do the cleanup. The code after
yield
runs at the end of the
with
block. The
yield
statement itself provides the object that will be assigned to the
with
target variable.

This implementation and the one that uses the context management protocol are practically equivalent. Depending on which one you find more readable, you might prefer one over the other. A downside of the function-based implementation is that it requires an understanding of advanced Python topics, such as decorators and generators.

The
@contextmanager
decorator reduces the boilerplate required to create a context manager. Instead of writing a whole class with
.__enter__()
and
.__exit__()
methods, you just need to implement a generator function with a single
yield
that produces whatever you want
.__enter__()
to return.

Opening Files for Writing: Second Version

You can use the
@contextmanager
to reimplement your
WritableFile
context manager. Here’s what rewriting it with this technique looks like:

>>> from contextlib import contextmanager >>> @contextmanager ... def writable_file(file_path): ... file = open(file_path, mode="w") ... try: ... yield file ... finally: ... file.close() ... >>> with writable_file("hello.txt") as file: ... file.write("Hello, World!") ...

In this case,
writable_file()
is a generator function that opens
file
for writing. Then it temporarily suspends its own execution and yields the resource so
with
can bind it to its target variable. When the flow of execution leaves the
with
code block, the function continues to execute and closes
file
correctly.

Mocking the Time

As a final example of how to create custom context managers with
@contextmanager
, say you’re testing a piece of code that works with time measurements. The code uses
time.time()
to get the current time measurement and do some further computations. Since time measurements vary, you decide to mock
time.time()
so you can test your code.

Here’s a function-based context manager that can help you do that:

>>> from contextlib import contextmanager >>> from time import time >>> @contextmanager ... def mock_time(): ... global time ... saved_time = time ... time = lambda: 42 ... yield ... time = saved_time ... >>> with mock_time(): ... print(f"Mocked time: {time()}") ... Mocked time: 42 >>> # Back to normal time >>> time() 1616075222.4410584

Inside
mock_time()
, you use a
global
statement to signal that you’re going to modify the global name
time
. Then you save the original
time()
function object in
saved_time
so you can safely restore it later. The next step is to monkey patch
time()
using a
lambda
function that always returns the same value,
42
.

The bare
yield
statement specifies that this context manager doesn’t have a useful object to send back to the
with
target variable for later use. After
yield
, you reset the global
time
to its original content.

When the execution enters the
with
block, any calls to
time()
return
42
. Once you leave the
with
code block, calls to
time()
return the expected current time. That’s it! Now you can test your time-related code.

Using Python context manager to implement the start and stop pattern

The following defines a
Timer
class that supports the context manager protocol:

from time import perf_counter class Timer: def __init__(self): self.elapsed = 0 def __enter__(self): self.start = perf_counter() return self def __exit__(self, exc_type, exc_value, exc_traceback): self.stop = perf_counter() self.elapsed = self.stop - self.start return False
Code language: Python (python)

How it works.

First, import the
perf_counter
from the
time
module.
Second, start the timer in the
__enter__()
method
Third, stop the timer in the
__exit__()
method and return the elapsed time.

Now, you can use the
Timer
class to measure the time needed to calculate the Fibonacci of 1000, one million times:

def fibonacci(n): f1 = 1 f2 = 1 for i in range(n-1): f1, f2 = f2, f1 + f2 return f1 with Timer() as timer: for _ in range(1, 1000000): fibonacci(1000) print(timer.elapsed)
Code language: Python (python)

If You’re Not Using Python DATA CLASSES Yet, You Should 🚀

Creating Custom Context Managers

You’ve already worked with context managers from the standard library and third-party libraries. There’s nothing special or magical about
open()
,
threading.Lock
,
decimal.localcontext()
, or the others. They just return objects that implement the context management protocol.

You can provide the same functionality by implementing both the
.__enter__()
and the
.__exit__()
special methods in your class-based context managers. You can also create custom function-based context managers using the
contextlib.contextmanager
decorator from the standard library and an appropriately coded generator function.

In general, context managers and the
with
statement aren’t limited to resource management. They allow you to provide and reuse common setup and teardown code. In other words, with context managers, you can perform any pair of operations that needs to be done before and after another operation or procedure, such as:

Open and close
Lock and release
Change and reset
Create and delete
Enter and exit
Start and stop
Setup and teardown

You can provide code to safely manage any of these pairs of operations in a context manager. Then you can reuse that context manager in
with
statements throughout your code. This prevents errors and reduces repetitive boilerplate code. It also makes your APIs safer, cleaner, and more user-friendly.

In the next two sections, you’ll learn the basics of creating class-based and function-based context managers.

Using the Python with Statement

As long as Python developers have incorporated the
with
statement into their coding practice, the tool has been shown to have several valuable use cases. More and more objects in the Python standard library now provide support for the context management protocol so you can use them in a
with
statement.

In this section, you’ll code some examples that show how to use the
with
statement with several classes both in the standard library and in third-party libraries.

Working With Files

So far, you’ve used
open()
to provide a context manager and manipulate files in a
with
construct. Opening files using the
with
statement is generally recommended because it ensures that open file descriptors are automatically closed after the flow of execution leaves the
with
code block.

As you saw before, the most common way to open a file using
with
is through the built-in
open()
:

with open("hello.txt", mode="w") as file: file.write("Hello, World!")

In this case, since the context manager closes the file after leaving the
with
code block, a common mistake might be the following:

>>> file = open("hello.txt", mode="w") >>> with file: ... file.write("Hello, World!") ... 13 >>> with file: ... file.write("Welcome to Real Python!") ... Traceback (most recent call last): File " ", line 1, in ValueError: I/O operation on closed file.

The first
with
successfully writes
"Hello, World!"
into
hello.txt
. Note that
.write()
returns the number of bytes written into the file,
13
. When you try to run a second
with
, however, you get a
ValueError
because your
file
is already closed.

Another way to use the
with
statement to open and manage files is by using
pathlib.Path.open()
:

>>> import pathlib >>> file_path = pathlib.Path("hello.txt") >>> with file_path.open("w") as file: ... file.write("Hello, World!") ... 13

Path
is a class that represents concrete paths to physical files in your computer. Calling
.open()
on a
Path
object that points to a physical file opens it just like
open()
would do. So,
Path.open()
works similarly to
open()
, but the file path is automatically provided by the
Path
object you call the method on.

Since
pathlib
provides an elegant, straightforward, and Pythonic way to manipulate file system paths, you should consider using
Path.open()
in your
with
statements as a best practice in Python.

Finally, whenever you load an external file, your program should check for possible issues, such as a missing file, writing and reading access, and so on. Here’s a general pattern that you should consider using when you’re working with files:

import pathlib import logging file_path = pathlib.Path("hello.txt") try: with file_path.open(mode="w") as file: file.write("Hello, World!") except OSError as error: logging.error("Writing to file %s failed due to: %s", file_path, error)

In this example, you wrap the
with
statement in a
try
…
except
statement. If an
OSError
occurs during the execution of
with
, then you use
logging
to log the error with a user-friendly and descriptive message.

Traversing Directories

The
os
module provides a function called
scandir()
, which returns an iterator over
os.DirEntry
objects corresponding to the entries in a given directory. This function is specially designed to provide optimal performance when you’re traversing a directory structure.

A call to
scandir()
with the path to a given directory as an argument returns an iterator that supports the context management protocol:

>>> import os >>> with os.scandir(".") as entries: ... for entry in entries: ... print(entry.name, "->", entry.stat().st_size, "bytes") ... Documents -> 4096 bytes Videos -> 12288 bytes Desktop -> 4096 bytes DevSpace -> 4096 bytes .profile -> 807 bytes Templates -> 4096 bytes Pictures -> 12288 bytes Public -> 4096 bytes Downloads -> 4096 bytes

In this example, you write a
with
statement with
os.scandir()
as the context manager supplier. Then you iterate over the entries in the selected directory (
"."
) and print their name and size on the screen. In this case,
.__exit__()
calls
scandir.close()
to close the iterator and release the acquired resources. Note that if you run this on your machine, you’ll get a different output depending on the content of your current directory.

Performing High-Precision Calculations

Unlike built-in floating-point numbers, the
decimal
module provides a way to adjust the precision to use in a given calculation that involves
Decimal
numbers. The precision defaults to
28
places, but you can change it to meet your problem requirements. A quick way to perform calculations with a custom precision is using
localcontext()
from
decimal
:

>>> from decimal import Decimal, localcontext >>> with localcontext() as ctx: ... ctx.prec = 42 ... Decimal("1") / Decimal("42") ... Decimal('0.0238095238095238095238095238095238095238095') >>> Decimal("1") / Decimal("42") Decimal('0.02380952380952380952380952381')

Here,
localcontext()
provides a context manager that creates a local decimal context and allows you to perform calculations using a custom precision. In the
with
code block, you need to set
.prec
to the new precision you want to use, which is
42
places in the example above. When the
with
code block finishes, the precision is reset back to its default value,
28
places.

Handling Locks in Multithreaded Programs

Another good example of using the
with
statement effectively in the Python standard library is
threading.Lock
. This class provides a primitive lock to prevent multiple threads from modifying a shared resource at the same time in a multithreaded application.

You can use a
Lock
object as the context manager in a
with
statement to automatically acquire and release a given lock. For example, say you need to protect the balance of a bank account:

import threading balance_lock = threading.Lock() # Use the try ... finally pattern balance_lock.acquire() try: # Update the account balance here ... finally: balance_lock.release() # Use the with pattern with balance_lock: # Update the account balance here ...

The
with
statement in the second example automatically acquires and releases a lock when the flow of execution enters and leaves the statement. This way, you can focus on what really matters in your code and forget about those repetitive operations.

In this example, the lock in the
with
statement creates a protected region known as the critical section, which prevents concurrent access to the account balance.

Testing for Exceptions With pytest

So far, you’ve coded a few examples using context managers that are available in the Python standard library. However, several third-party libraries include objects that support the context management protocol.

Say you’re testing your code with pytest. Some of your functions and code blocks raise exceptions under certain situations, and you want to test those cases. To do that, you can use
pytest.raises()
. This function allows you to assert that a code block or a function call raises a given exception.

Since
pytest.raises()
provides a context manager, you can use it in a
with
statement like this:

>>> import pytest >>> 1 / 0 Traceback (most recent call last): File " ", line 1, in ZeroDivisionError: division by zero >>> with pytest.raises(ZeroDivisionError): ... 1 / 0 ... >>> favorites = {"fruit": "apple", "pet": "dog"} >>> favorites["car"] Traceback (most recent call last): File " ", line 1, in KeyError: 'car' >>> with pytest.raises(KeyError): ... favorites["car"] ...

In the first example, you use
pytest.raises()
to capture the
ZeroDivisionError
that the expression
1 / 0
raises. The second example uses the function to capture the
KeyError
that is raised when you access a key that doesn’t exist in a given dictionary.

If your function or code block doesn’t raise the expected exception, then
pytest.raises()
raises a failure exception:

>>> import pytest >>> with pytest.raises(ZeroDivisionError): ... 4 / 2 ... 2.0 Traceback (most recent call last): ... Failed: DID NOT RAISE

Another cool feature of
pytest.raises()
is that you can specify a target variable to inspect the raised exception. For example, if you want to verify the error message, then you can do something like this:

>>> with pytest.raises(ZeroDivisionError) as exc: ... 1 / 0 ... >>> assert str(exc.value) == "division by zero"

You can use all these
pytest.raises()
features to capture the exceptions you raise from your functions and code block. This is a cool and useful tool that you can incorporate into your current testing strategy.

How To Create a Custom Context Manager in Python

The American theoretical physicist, Richard Feynman famously said —

What I cannot create, I do not understand.

So, to understand the functionalities of a context manager you must create one by yourself and there are two distinct ways of doing that.

The first one is a generator-based approach and the second one is a class-based approach. In this section, I’ll teach you both.

But before that, let me you a complex example that does more than merely opening and closing files in Python.

Imagine another Python application that must communicate with an SQLite database for reading and writing data.

You can write that program as follows:

import sqlite3 create_table_sql_statement = 'CREATE TABLE IF NOT EXISTS books(title TEXT, author TEXT)' insert_into_table_sql_statement = "INSERT INTO books VALUES ('If Tomorrow Comes', 'Sidney Sheldon'), ('The Lincoln Lawyer', 'Michael Connelly')" select_from_table_sql_statement = 'SELECT * FROM books' def main(): database_path = ':memory:' connection = sqlite3.connect(database_path) cursor = connection.cursor() try: cursor.execute(create_table_sql_statement) connection.commit() cursor.execute(insert_into_table_sql_statement) connection.commit() cursor.execute(select_from_table_sql_statement) print(cursor.fetchall()) except Exception as e: print(f'read or write action to the database failed: {e}') finally: connection.close() if __name__ == '__main__': main() # [('If Tomorrow Comes', 'Sidney Sheldon'), ('The Lincoln Lawyer', 'Michael Connelly')]

This Python program establishes a connection with an SQLite database. Then it creates a new table called books with two
TEXT
columns named
title
and
author
.

The program then stores information about three books on the table, retrieves them from the database, and prints out the retrieved data on the console.

As evident from the output of the
print()
statement, the program has successfully saved and retrieved the given data from the database.

There are three SQL queries in this program responsible for the database actions I just described.

create_table_sql_statement = 'CREATE TABLE IF NOT EXISTS books(title TEXT, author TEXT)' insert_into_table_sql_statement = "INSERT INTO books VALUES ('If Tomorrow Comes', 'Sidney Sheldon'), ('The Lincoln Lawyer', 'Michael Connelly')" select_from_table_sql_statement = 'SELECT * FROM books'

I’ve kept these three lines of code at the top of the file to keep the
main()
function to cleaner. The rest of the program sets up the database and executes the queries.

Python comes with excellent support for SQLite databases, thanks to the
sqlite3
module encapsulating useful methods such as the
sqlite3.connect()
method.

This method takes the path to a database as a string, attempts to establish a connection and in case of success, returns a
Connection
object.

If you pass
:memory:
instead of a file path, the program will create a temporary database on your computer’s memory.

Once you have a connection, you’ll need a
Cursor
object. A cursor object is a layer of abstraction required for executing SQL queries.

The
cursor()
method encapsulated within the
Connection
object returns a new cursor to the connected database.

Inside a
try
block, you can attempt to execute whatever query you want using the
execute()
or
executemany()
methods.

try: cursor.execute(create_table_sql_statement) connection.commit() cursor.execute(insert_into_table_sql_statement) connection.commit() cursor.execute(select_from_table_sql_statement) print(cursor.fetchall())

You need to call the
connection.commit()
method every time you write something to the database. Otherwise, the changes will be lost.

Data returned from a database remains within the
cursor
object and you can access them using the
cursor.fetchone()
or
cursor.fetchall()
methods.

In case of a failure, the
except
block will be triggered. The
finally
block will run unconditionally and close the database connection in the end.

This is fine and functional but like I’ve already said, it’s not very reusable and is error prone.

Unfortunately, or in our case fortunately Python doesn’t come with a built-in context manager for handling connections with SQLite databases.

So, let’s try and see if we can produce one ourselves.

How to Create a Class Based Context Manager in Python

To write a class-based context manager in Python, you need to create an empty class with three specific methods:

class Database: def __init__(self): pass def __enter__(self): pass def __exit__(self, exc_type, exc_val, exc_tb): pass

The first one is obviously the class constructor that doesn’t accept any parameter yet. It’ll be responsible for accepting a database path:

import sqlite3 class Database: def __init__(self, path: str): self.path = path def __enter__(self): pass def __exit__(self, exc_type, exc_val, exc_tb): pass

The
__enter__()
method handles the task of setting up the resource. This is where you establish the connection and instantiate the cursor:

import sqlite3 class Database: def __init__(self, path: str): self.path = path def __enter__(self): self.connection = sqlite3.connect(self.path) self.cursor = self.connection.cursor() return self def __exit__(self, exc_type, exc_val, exc_tb): pass

However you can not return two objects at once so you have to return the instance of the class itself.

Finally, the
__exit__()
method handles the task of closing the external resource in question.

import sqlite3 class Database: def __init__(self, path: str): self.path = path def __enter__(self): self.connection = sqlite3.connect(self.path) self.cursor = self.connection.cursor() return self def __exit__(self, exc_type, exc_val, exc_tb): if exc_type is not None: print(f'an error occurred: {exc_val}') self.connection.close()

You can use this context manager in conjunction with the
with
statement in your code as follows:

import sqlite3 create_table_sql_statement = 'CREATE TABLE IF NOT EXISTS books(title TEXT, author TEXT)' insert_into_table_sql_statement = "INSERT INTO books VALUES ('If Tomorrow Comes', 'Sidney Sheldon'), ('The Lincoln Lawyer', 'Michael Connelly')" select_from_table_sql_statement = 'SELECT * FROM books' class Database: def __init__(self, path: str): self.path = path def __enter__(self): self.connection = sqlite3.connect(self.path) self.cursor = self.connection.cursor() return self def __exit__(self, exc_type, exc_val, exc_tb): if exc_type is not None: print(f'an error occurred: {exc_val}') def main(): with Database(':memory:') as db: db.cursor.execute(create_table_sql_statement) db.connection.commit() db.cursor.execute(insert_into_table_sql_statement) db.connection.commit() db.cursor.execute(select_from_table_sql_statement) print(db.cursor.fetchall()) if __name__ == '__main__': main() # [('If Tomorrow Comes', 'Sidney Sheldon'), ('The Lincoln Lawyer', 'Michael Connelly')]

Evident from the output of the
print()
function call, the program has successfully stored and retrieved the given data from the database.

Without the
with
statement,
Database
is just a plain old class. However, the moment you put
with
infront of it, the three methods hop into action.

The
__init__()
method is the initializer and works identically to any other plain Python class’s initializer method. It takes the path to the database.

The
__enter__()
method sets up the connection to the database and returns the instance of the context manager class to the target variable,
db
in this case.

This target variable is now encapsulating both the connection and the cursor objects. You can access them as
db.connection
and
db.cursor
respectively.

Once the code inside the
with
block finishes running, the
__exit__()
method will execute and close the active connection to the database.

You can handle any exception that may occur during the execution inside the
__exit__()
method. If there is an exception,
exc_type
holds the type of the exception,
exc_val
holds the value of the exception,
exc_tb
holds the traceback.

If there is no exception, the three variables will have a value of
None
. I’ll not get into the details of exception handling in this article since that can take on many forms depending on what you’re dealing with.

To make this custom context manager accessible from anywhere in the program, you can put it into its own separate module or even package.

This is far better solution than the
try...except...finally
ladder you saw earlier. You don’t have to repeat yourself and chances of a human error is lower.

How to Create a Generator Based Context Manager in Python

Evident from the title of this section, this approach uses a generator instead of a class to implement a context manager.

Syntactically, generators are almost the same as normal functions, except that you need to use
yield
instead of
return
in a generator.

Writing a generator-based context manager requires less code but it also loses some of its readability.

You can write the generator-based equivalent of the class-based
Database
context manager as follows:

import sqlite3 from contextlib import contextmanager @contextmanager def database(path: str): connection = sqlite3.connect(path) try: cursor = connection.cursor() yield {'connection': connection, 'cursor': cursor} except Exception as e: print(f'an error occurred: {e}') finally: connection.close()

Instead of a class, you have a generator function here so there is no initializer. Instead, the function itself can accept the path to the database as a parameter.

Within a
try
block, you can establish a connection to the database, instantiate the cursor, and return both objects to the user.

You can write
yield connection, cursor
to return the two objects but in that case the generator will return them as a tuple.

I prefer to use strings over numbers as accessors and that’s why I have put the two objects inside a dictionary with descriptive keys.

The
except
block will run in case of an exception. Feel free to implement any exception handling strategy that you see fit.

The
finally
block will run unconditionally and close the open connection at the end of the
with
block.

Since there are no
__enter__()
or
__exit__()
methods either, you need to decorate the generator with the
@contextmanager
decorator.

This decorator defines a factory function for
with
statement context managers, without needing to create a class or separate
__enter__()
and
__exit__()
methods.

Usage of this context manager is identical to its class-based conterpart except the capitalization of its name.

import sqlite3 from contextlib import contextmanager create_table_sql_statement = 'CREATE TABLE IF NOT EXISTS books(title TEXT, author TEXT)' insert_into_table_sql_statement = "INSERT INTO books VALUES ('If Tomorrow Comes', 'Sidney Sheldon'), ('The Lincoln Lawyer', 'Michael Connelly')" select_from_table_sql_statement = 'SELECT * FROM books' @contextmanager def database(path: str): connection = sqlite3.connect(path) try: cursor = connection.cursor() yield {'connection': connection, 'cursor': cursor} except Exception as e: print(f'an error occurred: {e}') finally: connection.close() def main(): database_path = ':memory:' with database(database_path) as db: db.get('cursor').execute(create_table_sql_statement) db.get('connection').commit() db.get('cursor').execute(insert_into_table_sql_statement) db.get('connection').commit() db.get('cursor').execute(select_from_table_sql_statement) print(db.get('cursor').fetchall()) if __name__ == '__main__': main() # [('If Tomorrow Comes', 'Sidney Sheldon'), ('The Lincoln Lawyer', 'Michael Connelly')]

Since
db
is a dictionary instead of an object in this case, you will need to use square braces or the
get()
method to access the connection or cursor object.

Python3

class
ContextManager():

def
__init__(
self
):

print
'init method called'

def
__enter__(
self
):

print
'enter method called'

return
self

def
__exit__(
self
, exc_type, exc_value, exc_traceback):

print
'exit method called'

with ContextManager() as manager:

print
'with statement block'

Output:

init method called
enter method called
with statement block
exit method called

In this case, a ContextManager object is created. This is assigned to the variable after the keyword i.e manager. On running the above program, the following get executed in sequence:

__init__()
__enter__()
statement body (code inside the with block)
__exit__()[the parameters in this method are used to manage exceptions]

File management using context manager: Let’s apply the above concept to create a class that helps in file resource management. The FileManager class helps in opening a file, writing/reading contents, and then closing it.

Python - Class Based Context Manager — Python – Class Based Context Manager

27.Implementing a Context Manager as a Generator¶

We can also implement Context Managers using decorators and generators. Python has a contextlib module for this very purpose. Instead of a class, we can implement a Context Manager using a generator function. Let’s see a basic, useless example:

from contextlib import contextmanager @contextmanager def open_file(name): f = open(name, ‘w’) try: yield f finally: f.close()

Okay! This way of implementing Context Managers appear to be more intuitive and easy. However, this method requires some knowledge about generators, yield and decorators. In this example we have not caught any exceptions which might occur. It works in mostly the same way as the previous method.

Let’s dissect this method a little.

Python encounters the
yield
keyword. Due to this it creates a generator instead of a normal function.
Due to the decoration, contextmanager is called with the function
name (
open_file
) as its argument.
The
contextmanager
decorator returns the generator wrapped by the
GeneratorContextManager
object.
The
GeneratorContextManager
is assigned to the
open_file
function. Therefore, when we later call the
open_file
function, we are actually calling the
GeneratorContextManager
object.

So now that we know all this, we can use the newly generated Context Manager like this:

with open_file(‘some_file’) as f: f.write(‘hola!’)

The
with
statement in Python is a quite useful tool for properly managing external resources in your programs. It allows you to take advantage of existing context managers to automatically handle the setup and teardown phases whenever you’re dealing with external resources or with operations that require those phases.

Besides, the context management protocol allows you to create your own context managers so you can customize the way you deal with system resources. So, what’s the
with
statement good for?

In this tutorial, you’ll learn:

What the Python
with
statement is for and how to use it
What the context management protocol is
How to implement your own context managers

With this knowledge, you’ll write more expressive code and avoid resource leaks in your programs. The
with
statement helps you implement some common resource management patterns by abstracting their functionality and allowing them to be factored out and reused.

Free Download: Get a sample chapter from Python Tricks: The Book that shows you Python’s best practices with simple examples you can apply instantly to write more beautiful + Pythonic code.

Python3

with
open
("test.txt") as f:

data
f.read()

Let’s take the example of file management. When a file is opened, a file descriptor is consumed which is a limited resource. Only a certain number of files can be opened by a process at a time. The following program demonstrates it.

Context Managers in Python - Advanced Python 21 - Programming Tutorial — Context Managers in Python – Advanced Python 21 – Programming Tutorial

Keywords searched by users: python class context manager

Categories: Phổ biến 28 Python Class Context Manager

Python - Class Based Context Manager - Youtube — Python – Class Based Context Manager – Youtube

Python Tutorial: Context Managers - Efficiently Managing Resources - Youtube — Python Tutorial: Context Managers – Efficiently Managing Resources – Youtube

Python Timer Functions: Three Ways To Monitor Your Code – Real Python

Understanding “With” And Python’S Context Managers – Youtube

What Is A Context Manager? - Youtube — What Is A Context Manager? – Youtube

See more here: kientrucannam.vn

See more: https://kientrucannam.vn/vn/

Python Class Context Manager | Python Context Manager Protocol

Using the async with Statement

Managing Resources in Python

The try … finally Approach

The with Statement Approach

What is a Context Manager in Python?

27.Handling Exceptions¶

Why use a context manager

Summary

Summary

Conclusion

Summarizing the with Statement’s Advantages

Python context manager – bạn đã thực sự hiểu?

Quản lý tài nguyên

Quản lý tài nguyên hiệu quả hơn

Một số context manager hữu ích khác

Cài đặt context manager như một class

Xử lý exception

Sử dụng contextlib cài đặt context manager

foo

this is not html

this is another non-html content

Kết luận

Context Managers¶

Single use, reusable and reentrant context managers¶

Reentrant context managers¶

Reusable context managers¶

Context Managers in Python: Using the “with” statement

27.Implementing a Context Manager as a Class:¶

Python context manager applications

1) Open – Close

2) Lock – release

3) Start – stop

3) Change – reset

Coding Class-Based Context Managers

Writing a Sample Class-Based Context Manager

Handling Exceptions in a Context Manager

Opening Files for Writing: First Version

Redirecting the Standard Output

Measuring Execution Time

Python3

Conclusion

contextlib — Utilities for with-statement contexts¶

Implementing Python context manager protocol

Utilities¶

Introduction to Python context managers

Creating an Asynchronous Context Manager

Python with statement

The with statement

Examples and Recipes¶

Supporting a variable number of context managers¶

Catching exceptions from __enter__ methods¶

Cleaning up in an __enter__ implementation¶

Replacing any use of try-finally and flag variables¶

Using a context manager as a function decorator¶

Creating Context Managers

Function-Based Implementation

Class-based implementation

Explanation

Python3

Please Login to comment…

Writing Good APIs With Context Managers

Python context manager protocol

The __enter__() method

The __exit__() method

Python3

Creating Function-Based Context Managers

Opening Files for Writing: Second Version

Mocking the Time

Using Python context manager to implement the start and stop pattern

Creating Custom Context Managers

Using the Python with Statement

Working With Files

Traversing Directories

Performing High-Precision Calculations

Handling Locks in Multithreaded Programs

Testing for Exceptions With pytest

How To Create a Custom Context Manager in Python

How to Create a Class Based Context Manager in Python

How to Create a Generator Based Context Manager in Python

Catching exceptions from enter methods¶

Cleaning up in an enter implementation¶

The enter() method

The exit() method