Detailed Description

Low-level memory management support

The memory namespace provides functions and classes for interfacing with the seastar memory allocator.

The seastar memory allocator splits system memory into a pool per logical core (lcore). Memory allocated one an lcore should be freed on the same lcore; failing to do so carries a severe performance penalty. It is possible to share memory with another core, but this should be limited to avoid cache coherency traffic. You can obtain the memory layout of the current shard with get_memory_layout().

Critical allocation scopes

Seastar supports marking scopes as critical allocation scopes for the purpose of special treatment from various memory related utilities. See scoped_critical_alloc_section.

Diagnostics and debugging features

Allocation failure injector

Allows injecting allocation failures for testing resiliency against allocation failures, or exceptions in general. See:

Large allocation warning

Large allocations put great pressure on the allocator which might be unable to serve them even if there is enough memory available, due to memory fragmentation. This is especially relevant for long-running applications, the kind of applications that are typically built with seastar. This feature allows finding these large by logging a warning on large allocations, with the stacktrace of the. See:

Heap profiling

Heap profiling allows finding out how memory is used by your application, by recording the stacktrace of all allocations. See:

Abort on allocation failure

Often, the best way to debug an allocation failure is a coredump. This feature allows dumping core on allocation failures, containing the stack of the failed allocation, by means of aborting. To enable set the abort_on_seastar_bad_alloc configuration option or the respective command line flag.

Dump diagnostics report

Dump a diagnostic report of the state of the seastar allocator upon allocation failure. The report is dumped with the seastar_memory logger, with debug level. You can configure a report to be dumped with error level on certain allocation kinds, see:

The diagnostics report dump can be configured with the command line/configuration file via the dump-memory-diagnostics-on-alloc-failure-kind command-line flag/configuration item.

Classes
class	alloc_failure_injector

struct	disable_failure_guard

struct	memory_layout

class	scoped_critical_alloc_section
	Marks scopes that contain critical allocations. More...

class	scoped_heap_profiling

class	scoped_large_allocation_warning_disable
	Disable large allocation warnings for a scope. More...

class	scoped_large_allocation_warning_threshold
	Set a different large allocation warning threshold for a scope. More...

class	statistics
	Memory allocation statistics. More...

Typedefs
using	memory_diagnostics_writer = noncopyable_function< void(std::string_view)>
	A functor which writes its argument into the diagnostics report.

Enumerations
enum	alloc_failure_kind { alloc_failure_kind::none, alloc_failure_kind::critical, alloc_failure_kind::all }
	The kind of allocation failures to dump diagnostics report for. More...

Functions
statistics	stats ()
	Capture a snapshot of memory allocation statistics for this lcore.

memory::memory_layout	get_memory_layout ()

size_t	min_free_memory ()

void	set_min_free_pages (size_t pages)
	Sets the value of free memory low water mark in memory::page_size units.

void	set_large_allocation_warning_threshold (size_t threshold)

size_t	get_large_allocation_warning_threshold ()
	Gets the current large allocation warning threshold.

void	disable_large_allocation_warning ()
	Disable large allocation warnings.

void	set_heap_profiling_enabled (bool)

alloc_failure_injector &	local_failure_injector ()
	Return the shard-local alloc_failure_injector instance.

void	on_alloc_point ()
	Marks a point in code which should be considered for failure injection.

void	with_allocation_failures (noncopyable_function< void()> func)

bool	is_critical_alloc_section ()
	Is the current context inside a critical alloc section? More...

void	set_dump_memory_diagnostics_on_alloc_failure_kind (alloc_failure_kind)
	Configure when memory diagnostics are dumped. More...

void	set_dump_memory_diagnostics_on_alloc_failure_kind (std::string_view)
	Configure when memory diagnostics are dumped. More...

void	set_additional_diagnostics_producer (noncopyable_function< void(memory_diagnostics_writer)> producer)
	Set a producer of additional diagnostic information. More...

sstring	generate_memory_diagnostics_report ()

Class Documentation

◆ seastar::memory::disable_failure_guard

struct seastar::memory::disable_failure_guard

Class Members
scoped_critical_alloc_section	cs

◆ seastar::memory::memory_layout

struct seastar::memory::memory_layout

Class Members
uintptr_t	end
uintptr_t	start

Enumeration Type Documentation

◆ alloc_failure_kind

enum seastar::memory::alloc_failure_kind

strong

The kind of allocation failures to dump diagnostics report for.

Note that if the seastar_memory logger is set to level debug, there will be a report dumped for any allocation failure, regardless of this configuration.

Enumerator
none	Dump diagnostic error report for none of the allocation failures.
critical	Dump diagnostic error report for critical allocation failures, see scoped_critical_alloc_section.
all	Dump diagnostic error report for all the allocation failures.

Function Documentation

◆ generate_memory_diagnostics_report()

sstring seastar::memory::generate_memory_diagnostics_report ( )

Manually generate a diagnostics report

Note that contrary to the automated report generation (triggered by allocation failure), this method does allocate memory and can fail in low-memory conditions.

◆ is_critical_alloc_section()

bool seastar::memory::is_critical_alloc_section ( )

inline

Is the current context inside a critical alloc section?

Will return true if there is at least one scoped_critical_alloc_section alive in the current scope or the scope of any of the caller functions.

◆ min_free_memory()

size_t seastar::memory::min_free_memory ( )

Returns the value of free memory low water mark in bytes. When free memory is below this value, reclaimers are invoked until it goes above again.

◆ set_additional_diagnostics_producer()

void seastar::memory::set_additional_diagnostics_producer ( noncopyable_function< void(memory_diagnostics_writer)> producer )

Set a producer of additional diagnostic information.

This allows the application running on top of seastar to add its own part to the diagnostics dump. The application can supply higher level diagnostics information, that might help explain how the memory was consumed.

The application specific part will be added just below the main stats (free/used/total memory).

Parameters

producer - the functor to produce the additional diagnostics, specific to the application, to be added to the generated report. The producer is passed a writer functor, which it can use to add its parts to the report.

Note: As the report is generated at a time when allocations are failing, the producer should try as hard as possible to not allocate while producing the output.

◆ set_dump_memory_diagnostics_on_alloc_failure_kind() [1/2]

void seastar::memory::set_dump_memory_diagnostics_on_alloc_failure_kind ( alloc_failure_kind )

Configure when memory diagnostics are dumped.

See alloc_failure_kind on available options. Applies configuration on all shards.

◆ set_dump_memory_diagnostics_on_alloc_failure_kind() [2/2]

void seastar::memory::set_dump_memory_diagnostics_on_alloc_failure_kind ( std::string_view )

Configure when memory diagnostics are dumped.

String version. See alloc_failure_kind on available options. Applies configuration on all shards.

◆ set_heap_profiling_enabled()

void seastar::memory::set_heap_profiling_enabled ( bool )

Enable/disable heap profiling.

In order to use heap profiling you have to define SEASTAR_HEAPPROF. Heap profiling data is not currently exposed via an API for inspection, instead it was designed to be inspected from a debugger. For an example script that makes use of the heap profiling data see scylla-gdb.py This script can generate either textual representation of the data, or a zoomable flame graph (flame graph generation instructions, example flame graph).

◆ set_large_allocation_warning_threshold()

void seastar::memory::set_large_allocation_warning_threshold ( size_t threshold )

Enable the large allocation warning threshold.

Warn when allocation above a given threshold are performed.