GDB can be used to quickly and easily examine a core file that was produced when an execution crashed. It can give an approximate answer to the question of where the code was when the crash happened.
Examining a core file with GDB¶
GDB is available on all NERSC production systems in the default path. Run gdb with the following syntax:
nersc$ gdb executable-file core-file
After gdb starts use the "backtrace" command. For example:
nersc$ gdb mpi-hello core GNU gdb 6.6 Copyright (C) 2006 Free Software Foundation, Inc. GDB is free software, covered by the GNU General Public License, and you are welcome to change it and/or distribute copies of it under certain conditions. Type "show copying" to see the conditions. There is absolutely no warranty for GDB. Type "show warranty" for details. This GDB was configured as "x86_64-suse-linux"... Using host libthread_db library "/lib64/libthread_db.so.1". Core was generated by `./mpi-hello'. Program terminated with signal 8, Arithmetic exception. #0 0x0000000000400453 in MAIN () at ./mpi-hello.f:6 6 CALL MPI_INIT( ierr ) (gdb) backtrace #0 0x0000000000400453 in MAIN () at ./mpi-hello.f:6 #1 0x0000000000400330 in main () Current language: auto; currently fortran
Keep in mind that line number tracing when a code crashes is imprecise. The line number above about CALL
MPI_INIT() is probably not the line that the floating point exception occurred. Consult the gdb man pages for more information on this utility.