ns_crash - Platform-independent Command to Trigger a Segmentation Fault
The ns_crash command is intended for debugging purposes. It deliberately causes a segmentation violation (SEGV) in a platform-independent manner, ensuring that the server process is interrupted and a core dump is generated. This allows developers and system administrators to analyze the state of the server at the time of the crash using a C-level debugger, examining call stacks and variables as they were at the moment the command was issued.
Forces a segmentation fault, creating a core dump if the server is configured to allow it. The resulting core file can be analyzed using tools like gdb to investigate the cause and server state.
% ns_crash
When invoked, the above command will immediately cause the server to crash, generating a core dump (if allowed by the system and configuration). This can then be used to diagnose issues in server code or extensions running within NaviServer.
Core dump handling is highly platform and runtime dependent. Even when NaviServer restores the default signal disposition and re-raises a fatal signal, the location and availability of a resulting core file are controlled by the operating system, process limits, crash-reporting services, and, for containers, the surrounding container runtime. The following subsections are intended to ease system administraton and summarize the platform-specific setup needed to make ns_crash and other fatal signals produce useful crash diagnostics in practice.
When NaviServer runs inside a Linux container, core dump generation depends on both the host kernel settings and the container's runtime environment. In particular, the kernel's core_pattern is evaluated in the crashing process' mount namespace. Therefore, when core_pattern names an absolute path, that path must exist inside the container, not just on the host.
First, configure the host to write core files to a directory path rather than to a crash-handler pipe:
cat /proc/sys/kernel/core_pattern sudo mkdir -p /var/lib/naviserver/cores sudo chmod 1777 /var/lib/naviserver/cores echo '/var/lib/naviserver/cores/core.%e.%p.%t' | sudo tee /proc/sys/kernel/core_pattern
If the server changes user id or runs with elevated privileges, the Linux suid_dumpable setting may also be relevant:
cat /proc/sys/fs/suid_dumpable echo 2 | sudo tee /proc/sys/fs/suid_dumpable
The same directory must be visible inside the container at the same path. With Docker Compose, bind-mount the directory and allow unlimited core files:
services:
openacs:
init: true
volumes:
- /var/lib/naviserver/cores:/var/lib/naviserver/cores
ulimits:
core:
soft: -1
hard: -1
The init: true setting is important. Without it, the NaviServer process may run as PID 1 inside the container. Linux gives PID 1 special signal semantics, which can prevent the usual fatal-signal action from taking effect after a signal handler restores the default disposition and re-raises the signal. Running the container with an init process, or equivalently using docker run --init, ensures that nsd is not PID 1 and can terminate normally on fatal signals.
Inside the container, verify the effective core size limit and that the target directory exists:
ulimit -c ls -ld /var/lib/naviserver/cores
When a fatal signal occurs, NaviServer restores the signal's default disposition, unblocks the signal in the current thread, and re-raises it. If core dumps are enabled, the kernel should then write a file such as:
/var/lib/naviserver/cores/core.nsd.12345.1782400000
If no core file appears, check the following points:
ulimit -c inside the container must be unlimited or large enough.
The directory from /proc/sys/kernel/core_pattern must exist inside the container.
The directory must be writable by the user running NaviServer.
core_pattern must not point to a host-only path that is absent in the container.
NaviServer should not run as PID 1; use Docker init: true or --init.
Host crash handlers such as systemd-coredump or apport may intercept core dumps when core_pattern starts with a pipe character.
After a core file was written, inspect it from the host or from a suitable debug container with matching binaries and libraries:
gdb /path/to/nsd /var/lib/naviserver/cores/core.nsd.PID.TIME (gdb) thread apply all bt full
On macOS, invoking ns_crash may terminate the server and produce a CrashReporter report instead of a traditional core file. In this case, no file is written to the path configured via sysctl kern.corefile, even when ulimit reports that core dumps are enabled.
The crash report is typically written below:
~/Library/Logs/DiagnosticReports/
For example, after invoking ns_crash, look for a recent file whose name starts with "nsd-" and ends in ".ips":
ls -ltr ~/Library/Logs/DiagnosticReports/nsd-*.ips
These ".ips" files contain the macOS crash report, including the crashed thread and backtrace information available to the system crash reporter.
For interactive debugging on macOS, start NaviServer under lldb instead of relying on a traditional core file:
lldb -- /usr/local/ns/bin/nsd -f -t /path/to/nsd-config.tcl (lldb) run (lldb) thread backtrace all
When stopped at the crash in lldb, a core-like dump can be written explicitly with:
(lldb) process save-core /tmp/nsd.core
Before relying on traditional core files on macOS, verify the process limits and kernel settings in the same shell that starts NaviServer:
ulimit -S -c ulimit -H -c sysctl kern.coredump sysctl kern.corefile
Even with these settings enabled, recent macOS systems may still prefer CrashReporter ".ips" reports over writing a traditional core file.