An asynchronous function starts an operation which may continue long
after the function returns: The function itself returns a
future<T>
almost immediately, but it may take a while
until this future is resolved.
When such an asynchronous operation needs to operate on existing objects, or to use temporary objects, we need to worry about the lifetime of these objects: We need to ensure that these objects do not get destroyed before the asynchronous function completes (or it will try to use the freed object and malfunction or crash), and to also ensure that the object finally get destroyed when it is no longer needed (otherwise we will have a memory leak). Seastar offers a variety of mechanisms for safely and efficiently keeping objects alive for the right duration. In this section we will explore these mechanisms, and when to use each mechanism.
The most straightforward way to ensure that an object is alive when a
continuation runs and is destroyed afterwards is to pass its ownership
to the continuation. When continuation owns the object, the
object will be kept until the continuation runs, and will be destroyed
as soon as the continuation is not needed (i.e., it may have run, or
skipped in case of exception and then()
continuation).
We already saw above that the way for a continuation to get ownership of an object is through capturing:
::future<> slow_incr(int i) {
seastarreturn seastar::sleep(10ms).then([i] { return i + 1; });
}
Here the continuation captures the value of i
. In other
words, the continuation includes a copy of i
. When the
continuation runs 10ms later, it will have access to this value, and as
soon as the continuation finishes its object is destroyed, together with
its captured copy of i
. The continuation owns this copy of
i
.
Capturing by value as we did here - making a copy of the object we need in the continuation - is useful mainly for very small objects such as the integer in the previous example. Other objects are expensive to copy, or sometimes even cannot be copied. For example, the following is not a good idea:
::future<> slow_op(std::vector<int> v) {
seastar// this makes another copy of v:
return seastar::sleep(10ms).then([v] { /* do something with v */ });
}
This would be inefficient - as the vector v
, potentially
very long, will be copied and the copy will be saved in the
continuation. In this example, there is no reason to copy v
- it was anyway passed to the function by value and will not be used
again after capturing it into the continuation, as right after the
capture, the function returns and destroys its copy of
v
.
For such cases, C++14 allows moving the object into the continuation:
::future<> slow_op(std::vector<int> v) {
seastar// v is not copied again, but instead moved:
return seastar::sleep(10ms).then([v = std::move(v)] { /* do something with v */ });
}
Now, instead of copying the object v
into the
continuation, it is moved into the continuation. The
C++11-introduced move constructor moves the vector’s data into the
continuation and clears the original vector. Moving is a quick operation
- for a vector it only requires copying a few small fields such as the
pointer to the data. As before, once the continuation is dismissed the
vector is destroyed - and its data array (which was moved in the move
operation) is finally freed.
TODO: talk about temporary_buffer as an example of an object designed to be moved in this way.
In some cases, moving the object is undesirable. For example, some
code keeps references to an object or one of its fields and the
references become invalid if the object is moved. In some complex
objects, even the move constructor is slow. For these cases, C++
provides the useful wrapper std::unique_ptr<T>
. A
unique_ptr<T>
object owns an object of type
T
allocated on the heap. When a
unique_ptr<T>
is moved, the object of type T is not
touched at all - just the pointer to it is moved. An example of using
std::unique_ptr<T>
in capture is:
::future<> slow_op(std::unique_ptr<T> p) {
seastarreturn seastar::sleep(10ms).then([p = std::move(p)] { /* do something with *p */ });
}
std::unique_ptr<T>
is the standard C++ mechanism
for passing unique ownership of an object to a function: The object is
only owned by one piece of code at a time, and ownership is transferred
by moving the unique_ptr
object. A unique_ptr
cannot be copied: If we try to capture p by value, not by move, we will
get a compilation error.
The technique we described above - giving the continuation ownership of the object it needs to work on - is powerful and safe. But often it becomes hard and verbose to use. When an asynchronous operation involves not just one continuation but a chain of continuations that each needs to work on the same object, we need to pass the ownership of the object between each successive continuation, which can become inconvenient. It is especially inconvenient when we need to pass the same object into two separate asynchronous functions (or continuations) - after we move the object into one, the object needs to be returned so it can be moved again into the second. E.g.,
::future<> slow_op(T o) {
seastarreturn seastar::sleep(10ms).then([o = std::move(o)] {
// first continuation, doing something with o
...
// return o so the next continuation can use it!
return std::move(o);
}).then([](T o) {
// second continuation, doing something with o
...
});
}
This complexity arises because we wanted asynchronous functions and continuations to take the ownership of the objects they operated on. A simpler approach would be to have the caller of the asynchronous function continue to be the owner of the object, and just pass references to the object to the various other asynchronous functions and continuations which need the object. For example:
::future<> slow_op(T& o) { // <-- pass by reference
seastarreturn seastar::sleep(10ms).then([&o] {// <-- capture by reference
// first continuation, doing something with o
...
}).then([&o]) { // <-- another capture by reference
// second continuation, doing something with o
...
});
}
This approach raises a question: The caller of slow_op
is now responsible for keeping the object o
alive while the
asynchronous code started by slow_op
needs this object. But
how will this caller know how long this object is actually needed by the
asynchronous operation it started?
The most reasonable answer is that an asynchronous function may need access to its parameters until the future it returns is resolved - at which point the asynchronous code completes and no longer needs access to its parameters. We therefore recommend that Seastar code adopt the following convention:
Whenever an asynchronous function takes a parameter by reference, the caller must ensure that the referred object lives until the future returned by the function is resolved.
Note that this is merely a convention suggested by Seastar, and unfortunately nothing in the C++ language enforces it. C++ programmers in non-Seastar programs often pass large objects to functions as a const reference just to avoid a slow copy, and assume that the called function will not save this reference anywhere. But in Seastar code, that is a dangerous practice because even if the asynchronous function did not intend to save the reference anywhere, it may end up doing it implicitly by passing this reference to another function and eventually capturing it in a continuation.
It would be nice if future versions of C++ could help us catch incorrect uses of references. Perhaps we could have a tag for a special kind of reference, an “immediate reference” which a function can use use immediately (i.e, before returning a future), but cannot be captured into a continuation.
With this convention in place, it is easy to write complex
asynchronous functions functions like slow_op
which pass
the object around, by reference, until the asynchronous operation is
done. But how does the caller ensure that the object lives until the
returned future is resolved? The following is wrong:
::future<> f() {
seastar; // wrong! will be destroyed too soon!
T objreturn slow_op(obj);
}
It is wrong because the object obj
here is local to the
call of f
, and is destroyed as soon as f
returns a future - not when this returned future is resolved! The
correct thing for a caller to do would be to create the object
obj
on the heap (so it does not get destroyed as soon as
f
returns), and then run slow_op(obj)
and when
that future resolves (i.e., with .finally()
), destroy the
object.
Seastar provides a convenient idiom, do_with()
for doing
this correctly:
::future<> f() {
seastarreturn seastar::do_with(T(), [] (auto& obj) {
// obj is passed by reference to slow_op, and this is fine:
return slow_op(obj);
}
}
do_with
will do the given function
with the given object alive.
do_with
saves the given object on the heap, and calls
the given lambda with a reference to the new object. Finally it ensures
that the new object is destroyed after the returned future is resolved.
Usually, do_with is given an rvalue, i.e., an unnamed temporary
object or an std::move()
ed object, and do_with
moves that object into its final place on the heap. do_with
returns a future which resolves after everything described above is done
(the lambda’s future is resolved and the object is destroyed).
For convenience, do_with
can also be given multiple
objects to hold alive. For example here we create two objects and hold
alive them until the future resolves:
::future<> f() {
seastarreturn seastar::do_with(T1(), T2(), [] (auto& obj1, auto& obj2) {
return slow_op(obj1, obj2);
}
}
While do_with
can the lifetime of the objects it holds,
if the user accidentally makes copies of these objects, these copies
might have the wrong lifetime. Unfortunately, a simple typo like
forgetting an “&” can cause such accidental copies. For example, the
following code is broken:
::future<> f() {
seastarreturn seastar::do_with(T(), [] (T obj) { // WRONG: should be T&, not T
return slow_op(obj);
}
}
In this wrong snippet, obj
is mistakenly not a reference
to the object which do_with
allocated, but rather a copy of
it - a copy which is destroyed as soon as the lambda function returns,
rather than when the future it returns resolved. Such code will most
likely crash because the object is used after being freed. Unfortunately
the compiler will not warn about such mistakes. Users should get used to
always using the type “auto&” with do_with
- as in the
above correct examples - to reduce the chance of such mistakes.
For the same reason, the following code snippet is also wrong:
::future<> slow_op(T obj); // WRONG: should be T&, not T
seastar::future<> f() {
seastarreturn seastar::do_with(T(), [] (auto& obj) {
return slow_op(obj);
}
}
Here, although obj
was correctly passed to the lambda by
reference, we later accidentally passed slow_op()
a copy of
it (because here slow_op
takes the object by value, not by
reference), and this copy will be destroyed as soon as
slow_op
returns, not waiting until the returned future
resolves.
When using do_with
, always remember it requires adhering
to the convention described above: The asynchronous function which we
call inside do_with
must not use the objects held by
do_with
after the returned future is resolved. It
is a serious use-after-free bug for an asynchronous function to return a
future which resolves while still having background operations using the
do_with()
ed objects.
In general, it is rarely a good idea for an asynchronous function to
resolve while leaving behind background operations - even if those
operations do not use the do_with()
ed objects. Background
operations that we do not wait for may cause us to run out of memory (if
we don’t limit their number) and make it difficult to shut down the
application cleanly.
In the beginning of this chapter, we already noted that capturing a copy of an object into a continuation is the simplest way to ensure that the object is alive when the continuation runs and destroyed afterwards. However, complex objects are often expensive (in time and memory) to copy. Some objects cannot be copied at all, or are read-write and the continuation should modify the original object, not a new copy. The solution to all these issues are reference counted, a.k.a. shared objects:
A simple example of a reference-counted object in Seastar is a
seastar::file
, an object holding an open file object (we
will introduce seastar::file
in a later section). A
file
object can be copied, but copying does not involve
copying the file descriptor (let alone the file). Instead, both copies
point to the same open file, and a reference count is increased by 1.
When a file object is destroyed, the file’s reference count is decreased
by one, and only when the reference count reaches 0 the underlying file
is actually closed.
The fact that file
objects can be copied very quickly
and all copies actually point to the same file, make it very convenient
to pass them to asynchronous code; For example,
::future<uint64_t> slow_size(file f) {
seastarreturn seastar::sleep(10ms).then([f] {
return f.size();
});
}
Note how calling slow_size
is as simple as calling
slow_size(f)
, passing a copy of f
, without
needing to do anything special to ensure that f
is only
destroyed when no longer needed. That simply happens naturally when
nothing refers to f
any more.
You may wonder why return f.size()
in the above example
is safe: Doesn’t it start an asynchronous operation on f
(the file’s size may be stored on disk, so not immediately available),
and f
may be immediately destroyed when we return and
nothing keeps holding a copy of f
? If f
is
really the last reference, that is indeed a bug, but there is another
one: the file is never closed. The assumption that makes the code valid
is that there is another reference to f
that will be used
to close it. The close member function holds on to the reference count
of that object, so it continues to live even if nothing else keeps
holding on to it. Since all futures produced by a file object complete
before it is closed, all that is needed for correctness is to remember
to always close files.
The reference counting has a run-time cost, but it is usually very
small; It is important to remember that Seastar objects are always used
by a single CPU only, so the reference-count increment and decrement
operations are not the slow atomic operations often used for reference
counting, but just regular CPU-local integer operations. Moreover,
judicious use of std::move()
and the compiler’s optimizer
can reduce the number of unnecessary back-and-forth increment and
decrement of the reference count.
C++11 offers a standard way of creating reference-counted shared
objects - using the template std::shared_ptr<T>
. A
shared_ptr
can be used to wrap any type into a
reference-counted shared object like seastar::file
above.
However, the standard std::shared_ptr
was designed with
multi-threaded applications in mind so it uses slow atomic
increment/decrement operations for the reference count which we already
noted is unnecessary in Seastar. For this reason Seastar offers its own
single-threaded implementation of this template,
seastar::shared_ptr<T>
. It is similar to
std::shared_ptr<T>
except no atomic operations are
used.
Additionally, Seastar also provides an even lower overhead variant of
shared_ptr
: seastar::lw_shared_ptr<T>
.
The full-featured shared_ptr
is complicated by the need to
support polymorphic types correctly (a shared object created of one
class, and accessed through a pointer to a base class). It makes
shared_ptr
need to add two words to the shared object, and
two words to each shared_ptr
copy. The simplified
lw_shared_ptr
- which does not support
polymorphic types - adds just one word in the object (the reference
count) and each copy is just one word - just like copying a regular
pointer. For this reason, the light-weight
seastar::lw_shared_ptr<T>
should be preferred when
possible (T
is not a polymorphic type), otherwise
seastar::shared_ptr<T>
. The slower
std::shared_ptr<T>
should never be used in sharded
Seastar applications.
Wouldn’t it be convenient if we could save objects on a stack just like we normally do in synchronous code? I.e., something like:
int i = ...;
::sleep(10ms).get();
seastarreturn i;
Seastar allows writing such code, by using a
seastar::thread
object which comes with its own stack. A
complete example using a seastar::thread
might look like
this:
::future<> slow_incr(int i) {
seastarreturn seastar::async([i] {
::sleep(10ms).get();
seastar// We get here after the 10ms of wait, i is still available.
return i + 1;
});
}
We present seastar::thread
,
seastar::async()
and seastar::future::get()
in
the seastar::thread section.