systemd-analyze — Analyze and debug system manager
Examples (TL;DR)
-
List time of each unit to start up:
systemd-analyze blame
-
Print a tree of the time critical chain of units:
systemd-analyze critical-chain
Synopsis
systemd-analyze [Options...] [time]
systemd-analyze [Options...] blame
systemd-analyze [Options...] critical-chain [UNIT...]
systemd-analyze [Options...] dump
systemd-analyze [Options...] plot [>file.svg]
systemd-analyze [Options...] dot [PATTERN...] [>file.dot]
systemd-analyze [Options...] unit-paths
systemd-analyze [Options...] exit-status [STATUS...]
systemd-analyze [Options...] condition CONDITION...
systemd-analyze [Options...] syscall-filter [SET...]
systemd-analyze [Options...] calendar SPEC...
systemd-analyze [Options...] timestamp TIMESTAMP...
systemd-analyze [Options...] timespan SPAN...
systemd-analyze [Options...] cat-config NAME|PATH...
systemd-analyze [Options...] verify [FILE...]
systemd-analyze [Options...] security UNIT...
Description
systemd-analyze may be used to determine system boot-up performance statistics and retrieve other state and tracing information from the system and service manager, and to verify the correctness of unit files. It is also used to access special functions useful for advanced system manager debugging.
If no command is passed, systemd-analyze time is implied.
systemd-analyze time
This command prints the time spent in the kernel before userspace has been reached, the time spent in the initial RAM disk (initrd) before normal system userspace has been reached, and the time normal system userspace took to initialize. Note that these measurements simply measure the time passed up to the point where all system services have been spawned, but not necessarily until they fully finished initialization or the disk is idle.
Example 1. Show how long the boot took
# in a container $ systemd-analyze time Startup finished in 296ms (userspace) multi-user.target reached after 275ms in userspace # on a real machine $ systemd-analyze time Startup finished in 2.584s (kernel) + 19.176s (initrd) + 47.847s (userspace) = 1min 9.608s multi-user.target reached after 47.820s in userspace
systemd-analyze blame
This command prints a list of all running units, ordered by the time they took to initialize. This information may be used to optimize boot-up times. Note that the output might be misleading as the initialization of one service might be slow simply because it waits for the initialization of another service to complete. Also note: systemd-analyze blame doesn't display results for services with Type=simple, because systemd considers such services to be started immediately, hence no measurement of the initialization delays can be done. Also note that this command only shows the time units took for starting up, it does not show how long unit jobs spent in the execution queue. In particular it shows the time units spent in "activating" state, which is not defined for units such as device units that transition directly from "inactive" to "active". This command hence gives an impression of the performance of program code, but cannot accurately reflect latency introduced by waiting for hardware and similar events.
Example 2. Show which units took the most time during boot
$ systemd-analyze blame 32.875s pmlogger.service 20.905s systemd-networkd-wait-online.service 13.299s dev-vda1.device ... 23ms sysroot.mount 11ms initrd-udevadm-cleanup-db.service 3ms sys-kernel-config.mount
systemd-analyze critical-chain [UNIT...]
This command prints a tree of the time-critical chain of units (for each of the specified UNITs or for the default target otherwise). The time after the unit is active or started is printed after the "@" character. The time the unit takes to start is printed after the "+" character. Note that the output might be misleading as the initialization of services might depend on socket activation and because of the parallel execution of units. Also, similar to the blame command, this only takes into account the time units spent in "activating" state, and hence does not cover units that never went through an "activating" state (such as device units that transition directly from "inactive" to "active"). Moreover it does not show information on jobs (and in particular not jobs that timed out).
Example 3. systemd-analyze time
$ systemd-analyze critical-chain multi-user.target @47.820s └─pmie.service @35.968s +548ms └─pmcd.service @33.715s +2.247s └─network-online.target @33.712s └─systemd-networkd-wait-online.service @12.804s +20.905s └─systemd-networkd.service @11.109s +1.690s └─systemd-udevd.service @9.201s +1.904s └─systemd-tmpfiles-setup-dev.service @7.306s +1.776s └─kmod-static-nodes.service @6.976s +177ms └─systemd-journald.socket └─system.slice └─-.slice
systemd-analyze dump
This command outputs a (usually very long) human-readable serialization of the complete server state. Its format is subject to change without notice and should not be parsed by applications.
Example 4. Show the internal state of user manager
$ systemd-analyze --user dump Timestamp userspace: Thu 2019-03-14 23:28:07 CET Timestamp finish: Thu 2019-03-14 23:28:07 CET Timestamp generators-start: Thu 2019-03-14 23:28:07 CET Timestamp generators-finish: Thu 2019-03-14 23:28:07 CET Timestamp units-load-start: Thu 2019-03-14 23:28:07 CET Timestamp units-load-finish: Thu 2019-03-14 23:28:07 CET -> Unit proc-timer_list.mount: Description: /proc/timer_list ... -> Unit default.target: Description: Main user target ...
systemd-analyze plot
This command prints an SVG graphic detailing which system services have been started at what time, highlighting the time they spent on initialization.
Example 5. Plot a bootchart
$ systemd-analyze plot >bootup.svg $ eog bootup.svg&
systemd-analyze dot [pattern...]
This command generates textual dependency graph description in dot format for further processing with the GraphViz dot(1) tool. Use a command line like systemd-analyze dot | dot -Tsvg >systemd.svg to generate a graphical dependency tree. Unless --order or --require is passed, the generated graph will show both ordering and requirement dependencies. Optional pattern globbing style specifications (e.g. *.target) may be given at the end. A unit dependency is included in the graph if any of these patterns match either the origin or destination node.
Example 6. Plot all dependencies of any unit whose name starts with "avahi-daemon"
$ systemd-analyze dot 'avahi-daemon.*' | dot -Tsvg >avahi.svg $ eog avahi.svg
Example 7. Plot the dependencies between all known target units
$ systemd-analyze dot --to-pattern='*.target' --from-pattern='*.target' \ | dot -Tsvg >targets.svg $ eog targets.svg
systemd-analyze unit-paths
This command outputs a list of all directories from which unit files, .d overrides, and .wants, .requires symlinks may be loaded. Combine with --user to retrieve the list for the user manager instance, and --global for the global configuration of user manager instances.
Example 8. Show all paths for generated units
$ systemd-analyze unit-paths | grep '^/run' /run/systemd/system.control /run/systemd/transient /run/systemd/generator.early /run/systemd/system /run/systemd/system.attached /run/systemd/generator /run/systemd/generator.late
Note that this verb prints the list that is compiled into systemd-analyze itself, and does not communicate with the running manager. Use
systemctl [--user] [--global] show -p UnitPath --value
to retrieve the actual list that the manager uses, with any empty directories omitted.
systemd-analyze exit-status [STATUS...]
This command prints a list of exit statuses along with their "class", i.e. the source of the definition (one of "glibc", "systemd", "LSB", or "BSD"), see the Process Exit Codes section in systemd.exec(5). If no additional arguments are specified, all known statuses are are shown. Otherwise, only the definitions for the specified codes are shown.
Example 9. Show some example exit status names
$ systemd-analyze exit-status 0 1 {63..65} NAME STATUS CLASS SUCCESS 0 glibc FAILURE 1 glibc - 63 - USAGE 64 BSD DATAERR 65 BSD
systemd-analyze condition CONDITION...
This command will evaluate Condition*=... and Assert*=... assignments, and print their values, and the resulting value of the combined condition set. See systemd.unit(5) for a list of available conditions and asserts.
Example 10. Evaluate conditions that check kernel versions
$ systemd-analyze condition 'ConditionKernelVersion = ! <4.0' \ 'ConditionKernelVersion = >=5.1' \ 'ConditionACPower=|false' \ 'ConditionArchitecture=|!arm' \ 'AssertPathExists=/etc/os-release' test.service: AssertPathExists=/etc/os-release succeeded. Asserts succeeded. test.service: ConditionArchitecture=|!arm succeeded. test.service: ConditionACPower=|false failed. test.service: ConditionKernelVersion=>=5.1 succeeded. test.service: ConditionKernelVersion=!<4.0 succeeded. Conditions succeeded.
systemd-analyze syscall-filter [SET...]
This command will list system calls contained in the specified system call set SET, or all known sets if no sets are specified. Argument SET must include the "@" prefix.
systemd-analyze calendar EXPRESSION...
This command will parse and normalize repetitive calendar time events, and will calculate when they elapse next. This takes the same input as the OnCalendar= setting in systemd.timer(5), following the syntax described in systemd.time(7). By default, only the next time the calendar expression will elapse is shown; use --iterations= to show the specified number of next times the expression elapses. Each time the expression elapses forms a timestamp, see the timestamp verb below.
Example 11. Show leap days in the near future
$ systemd-analyze calendar --iterations=5 '*-2-29 0:0:0' Original form: *-2-29 0:0:0 Normalized form: *-02-29 00:00:00 Next elapse: Sat 2020-02-29 00:00:00 UTC From now: 11 months 15 days left Iter. #2: Thu 2024-02-29 00:00:00 UTC From now: 4 years 11 months left Iter. #3: Tue 2028-02-29 00:00:00 UTC From now: 8 years 11 months left Iter. #4: Sun 2032-02-29 00:00:00 UTC From now: 12 years 11 months left Iter. #5: Fri 2036-02-29 00:00:00 UTC From now: 16 years 11 months left
systemd-analyze timestamp TIMESTAMP...
This command parses a timestamp (i.e. a single point in time) and outputs the normalized form and the difference between this timestamp and now. The timestamp should adhere to the syntax documented in systemd.time(7), section "PARSING TIMESTAMPS".
Example 12. Show parsing of timestamps
$ systemd-analyze timestamp yesterday now tomorrow Original form: yesterday Normalized form: Mon 2019-05-20 00:00:00 CEST (in UTC): Sun 2019-05-19 22:00:00 UTC UNIX seconds: @15583032000 From now: 1 day 9h ago Original form: now Normalized form: Tue 2019-05-21 09:48:39 CEST (in UTC): Tue 2019-05-21 07:48:39 UTC UNIX seconds: @1558424919.659757 From now: 43us ago Original form: tomorrow Normalized form: Wed 2019-05-22 00:00:00 CEST (in UTC): Tue 2019-05-21 22:00:00 UTC UNIX seconds: @15584760000 From now: 14h left
systemd-analyze timespan EXPRESSION...
This command parses a time span (i.e. a difference between two timestamps) and outputs the normalized form and the equivalent value in microseconds. The time span should adhere to the syntax documented in systemd.time(7), section "PARSING TIME SPANS". Values without units are parsed as seconds.
Example 13. Show parsing of timespans
$ systemd-analyze timespan 1s 300s '1year 0.000001s' Original: 1s μs: 1000000 Human: 1s Original: 300s μs: 300000000 Human: 5min Original: 1year 0.000001s μs: 31557600000001 Human: 1y 1us
systemd-analyze cat-config NAME|PATH...
This command is similar to systemctl cat, but operates on config files. It will copy the contents of a config file and any drop-ins to standard output, using the usual systemd set of directories and rules for precedence. Each argument must be either an absolute path including the prefix (such as /etc/systemd/logind.conf or /usr/lib/systemd/logind.conf), or a name relative to the prefix (such as systemd/logind.conf).
Example 14. Showing logind configuration
$ systemd-analyze cat-config systemd/logind.conf # /etc/systemd/logind.conf ... [Login] NAutoVTs=8 ... # /usr/lib/systemd/logind.conf.d/20-test.conf ... some override from another package # /etc/systemd/logind.conf.d/50-override.conf ... some administrator override
systemd-analyze verify FILE...
This command will load unit files and print warnings if any errors are detected. Files specified on the command line will be loaded, but also any other units referenced by them. The full unit search path is formed by combining the directories for all command line arguments, and the usual unit load paths (variable $SYSTEMD_UNIT_PATH is supported, and may be used to replace or augment the compiled in set of unit load paths; see systemd.unit(5)). All units files present in the directories containing the command line arguments will be used in preference to the other paths.
The following errors are currently detected:
- unknown sections and directives,
- missing dependencies which are required to start the given unit,
- man pages listed in Documentation= which are not found in the system,
- commands listed in ExecStart= and similar which are not found in the system or not executable.
Example 15. Misspelt directives
$ cat ./user.slice [Unit] WhatIsThis=11 Documentation=man:nosuchfile(1) Requires=different.service [Service] Description=x $ systemd-analyze verify ./user.slice [./user.slice:9] Unknown lvalue 'WhatIsThis' in section 'Unit' [./user.slice:13] Unknown section 'Service'. Ignoring. Error: org.freedesktop.systemd1.LoadFailed: Unit different.service failed to load: No such file or directory. Failed to create user.slice/start: Invalid argument user.slice: man nosuchfile(1) command failed with code 16
Example 16. Missing service units
$ tail ./a.socket ./b.socket ==> ./a.socket <== [Socket] ListenStream=100 ==> ./b.socket <== [Socket] ListenStream=100 Accept=yes $ systemd-analyze verify ./a.socket ./b.socket Service a.service not loaded, a.socket cannot be started. Service b@0.service not loaded, b.socket cannot be started.
systemd-analyze security [UNIT...]
This command analyzes the security and sandboxing settings of one or more specified service units. If at least one unit name is specified the security settings of the specified service units are inspected and a detailed analysis is shown. If no unit name is specified, all currently loaded, long-running service units are inspected and a terse table with results shown. The command checks for various security-related service settings, assigning each a numeric "exposure level" value, depending on how important a setting is. It then calculates an overall exposure level for the whole unit, which is an estimation in the range 0.0...10.0 indicating how exposed a service is security-wise. High exposure levels indicate very little applied sandboxing. Low exposure levels indicate tight sandboxing and strongest security restrictions. Note that this only analyzes the per-service security features systemd itself implements. This means that any additional security mechanisms applied by the service code itself are not accounted for. The exposure level determined this way should not be misunderstood: a high exposure level neither means that there is no effective sandboxing applied by the service code itself, nor that the service is actually vulnerable to remote or local attacks. High exposure levels do indicate however that most likely the service might benefit from additional settings applied to them.
Please note that many of the security and sandboxing settings individually can be circumvented — unless combined with others. For example, if a service retains the privilege to establish or undo mount points many of the sandboxing options can be undone by the service code itself. Due to that is essential that each service uses the most comprehensive and strict sandboxing and security settings possible. The tool will take into account some of these combinations and relationships between the settings, but not all. Also note that the security and sandboxing settings analyzed here only apply to the operations executed by the service code itself. If a service has access to an IPC system (such as D-Bus) it might request operations from other services that are not subject to the same restrictions. Any comprehensive security and sandboxing analysis is hence incomplete if the IPC access policy is not validated too.
Example 17. Analyze systemd-logind.service
$ systemd-analyze security --no-pager systemd-logind.service NAME DESCRIPTION EXPOSURE ✗ PrivateNetwork= Service has access to the host's network 0.5 ✗ User=/DynamicUser= Service runs as root user 0.4 ✗ DeviceAllow= Service has no device ACL 0.2 ✓ IPAddressDeny= Service blocks all IP address ranges ... → Overall exposure level for systemd-logind.service: 4.1 OK 🙂
Options
The following options are understood:
- --system
Operates on the system systemd instance. This is the implied default.
- --user
Operates on the user systemd instance.
- --global
Operates on the system-wide configuration for user systemd instance.
- --order, --require
When used in conjunction with the dot command (see above), selects which dependencies are shown in the dependency graph. If --order is passed, only dependencies of type After= or Before= are shown. If --require is passed, only dependencies of type Requires=, Requisite=, Wants= and Conflicts= are shown. If neither is passed, this shows dependencies of all these types.
- --from-pattern=, --to-pattern=
-
When used in conjunction with the dot command (see above), this selects which relationships are shown in the dependency graph. Both options require a glob(7) pattern as an argument, which will be matched against the left-hand and the right-hand, respectively, nodes of a relationship.
Each of these can be used more than once, in which case the unit name must match one of the values. When tests for both sides of the relation are present, a relation must pass both tests to be shown. When patterns are also specified as positional arguments, they must match at least one side of the relation. In other words, patterns specified with those two options will trim the list of edges matched by the positional arguments, if any are given, and fully determine the list of edges shown otherwise.
- --fuzz=timespan
When used in conjunction with the critical-chain command (see above), also show units, which finished timespan earlier, than the latest unit in the same level. The unit of timespan is seconds unless specified with a different unit, e.g. "50ms".
- --man=no
Do not invoke man to verify the existence of man pages listed in Documentation=.
- --generators
Invoke unit generators, see systemd.generator(7). Some generators require root privileges. Under a normal user, running with generators enabled will generally result in some warnings.
- --root=PATH
With cat-files, show config files underneath the specified root path PATH.
- --iterations=NUMBER
When used with the calendar command, show the specified number of iterations the specified calendar expression will elapse next. Defaults to 1.
- --base-time=TIMESTAMP
When used with the calendar command, show next iterations relative to the specified point in time. If not specified defaults to the current time.
- -H, --host=
Execute the operation remotely. Specify a hostname, or a username and hostname separated by "@", to connect to. The hostname may optionally be suffixed by a port ssh is listening on, separated by ":", and then a container name, separated by "/", which connects directly to a specific container on the specified host. This will use SSH to talk to the remote machine manager instance. Container names may be enumerated with machinectl -H HOST. Put IPv6 addresses in brackets.
- -M, --machine=
Execute operation on a local container. Specify a container name to connect to.
- -h, --help
Print a short help text and exit.
- --version
Print a short version string and exit.
- --no-pager
Do not pipe output into a pager.
Exit Status
On success, 0 is returned, a non-zero failure code otherwise.
Environment
- $SYSTEMD_PAGER
Pager to use when --no-pager is not given; overrides $PAGER. If neither $SYSTEMD_PAGER nor $PAGER are set, a set of well-known pager implementations are tried in turn, including less(1) and more(1), until one is found. If no pager implementation is discovered no pager is invoked. Setting this environment variable to an empty string or the value "cat" is equivalent to passing --no-pager.
- $SYSTEMD_LESS
-
Override the options passed to less (by default "FRSXMK").
Users might want to change two options in particular:
- K
-
This option instructs the pager to exit immediately when Ctrl+C is pressed. To allow less to handle Ctrl+C itself to switch back to the pager command prompt, unset this option.
If the value of $SYSTEMD_LESS does not include "K", and the pager that is invoked is less, Ctrl+C will be ignored by the executable, and needs to be handled by the pager.
- X
This option instructs the pager to not send termcap initialization and deinitialization strings to the terminal. It is set by default to allow command output to remain visible in the terminal even after the pager exits. Nevertheless, this prevents some pager functionality from working, in particular paged output cannot be scrolled with the mouse.
See less(1) for more discussion.
- $SYSTEMD_LESSCHARSET
Override the charset passed to less (by default "utf-8", if the invoking terminal is determined to be UTF-8 compatible).
- $SYSTEMD_COLORS
The value must be a boolean. Controls whether colorized output should be generated. This can be specified to override the decision that systemd makes based on $TERM and what the console is connected to.
- $SYSTEMD_URLIFY
The value must be a boolean. Controls whether clickable links should be generated in the output for terminal emulators supporting this. This can be specified to override the decision that systemd makes based on $TERM and other conditions.
See Also
Referenced By
systemd-boot(7), systemd.directives(7), systemd.exec(5), systemd.index(7), systemd-nspawn(1), systemd.time(7), systemd.unit(5).