TEST!!! - kdump on Oracle Linux (Unbreakable Enterprise Kernel)

I posted Just learn Kdump. That is how to setup kdump on linux. Today, I installed Oracle Linux 6.1 and It's Unbreakable Enterprise Kernel (2.6.32-100.34.1.el6uek.x86_64), then I tested kdump.

[root@mytest ~]# rpm -q kexec-tools
kexec-tools-2.0.0-188.0.2.el6.x86_64
[root@mytest ~]# grep CONFIG_KEXEC /boot/config-2.6.32-100.34.1.el6uek.x86_64
CONFIG_KEXEC=y
[root@mytest ~]# grep CONFIG_CRASH_DUMP /boot/config-2.6.32-100.34.1.el6uek.x86_64
CONFIG_CRASH_DUMP=y
[root@mytest ~]# cat /etc/grub.conf | grep `uname -r`
title Oracle Linux Server (2.6.32-100.34.1.el6uek.x86_64)
kernel /boot/vmlinuz-2.6.32-100.34.1.el6uek.x86_64 ro root=UUID=f52aec57-89fc-40b9-a8a8-1bf32694d05f rd_NO_LUKS rd_NO_LVM rd_NO_MD rd_NO_DM LANG=en_US.UTF-8 SYSFONT=latarcyrheb-sun16 KEYBOARDTYPE=pc KEYTABLE=us rhgb quiet
initrd /boot/initramfs-2.6.32-100.34.1.el6uek.x86_64.img

Changed to set crashkernel.

[root@mytest ~]# cat /etc/grub.conf | grep `uname -r`
title Oracle Linux Server (2.6.32-100.34.1.el6uek.x86_64)
kernel /boot/vmlinuz-2.6.32-100.34.1.el6uek.x86_64 ro root=UUID=f52aec57-89fc-40b9-a8a8-1bf32694d05f rd_NO_LUKS rd_NO_LVM rd_NO_MD rd_NO_DM LANG=en_US.UTF-8 SYSFONT=latarcyrheb-sun16 KEYBOARDTYPE=pc KEYTABLE=us rhgb quiet crashkernel=128M@16M
initrd /boot/initramfs-2.6.32-100.34.1.el6uek.x86_64.img

[root@mytest ~]# runlevel
N 3
[root@mytest ~]# chkconfig --level 3 kdump on
[root@mytest ~]# reboot

After rebooted.

[root@mytest ~]# cat /proc/cmdline
ro root=UUID=f52aec57-89fc-40b9-a8a8-1bf32694d05f rd_NO_LUKS rd_NO_LVM rd_NO_MD rd_NO_DM LANG=en_US.UTF-8 SYSFONT=latarcyrheb-sun16 KEYBOARDTYPE=pc KEYTABLE=us rhgb quiet crashkernel=128M@16M
[root@mytest ~]# /etc/init.d/kdump status
Kdump is not operational
[root@mytest ~]# /etc/init.d/kdump start
Starting kdump: [FAILED]
[root@mytest ~]# dmesg | grep -i crashkernel
Command line: ro root=UUID=f52aec57-89fc-40b9-a8a8-1bf32694d05f rd_NO_LUKS rd_NO_LVM rd_NO_MD rd_NO_DM LANG=en_US.UTF-8 SYSFONT=latarcyrheb-sun16 KEYBOARDTYPE=pc KEYTABLE=us rhgb quiet crashkernel=128M@16M
crashkernel reservation failed - memory is in use
Kernel command line: ro root=UUID=f52aec57-89fc-40b9-a8a8-1bf32694d05f rd_NO_LUKS rd_NO_LVM rd_NO_MD rd_NO_DM LANG=en_US.UTF-8 SYSFONT=latarcyrheb-sun16 KEYBOARDTYPE=pc KEYTABLE=us rhgb quiet crashkernel=128M@16M

Minimum value for crashkernel option is 128M@16M ??? Yes, but not for Unbreakable Enterprise Kernel. It needs Higher memory!!! Increased the second value (32M). OK... changed 16M to 32M and rebooted.

[root@mytest ~]# /etc/init.d/kdump status
Kdump is not operational
[root@mytest ~]# /etc/init.d/kdump start
Starting kdump: [FAILED]

Still not work!!! Increased the second value (64M) for Unbreakable Enterprise Kernel (2.6.32-100.34.1.el6uek.x86_64), then rebooted

[root@mytest ~]# /etc/init.d/kdump status
Kdump is operational
[root@mytest ~]# cat /proc/cmdline
ro root=UUID=f52aec57-89fc-40b9-a8a8-1bf32694d05f rd_NO_LUKS rd_NO_LVM rd_NO_MD rd_NO_DM LANG=en_US.UTF-8 SYSFONT=latarcyrheb-sun16 KEYBOARDTYPE=pc KEYTABLE=us rhgb quiet crashkernel=128M@64M

It's work!!! If You would like to use KDUMP with Unbreakable Enterprise Kernel. look at crashkernel=128M@64M. then TEST!!!

[root@mytest ~]# echo c > /proc/sysrq-trigger

System panic and rebooted. (TEST ON TEST SYSTEM ONLY)

[root@mytest ~]# last
reboot system boot 2.6.32-100.34.1. Tue Aug 23 10:47 - 10:48 (00:00)

[root@mytest ~]# ls -l /var/crash/127.0.0.1-2011-08-23-10\:46\:22/vmcore
-rw-------. 1 root root 34659098 Aug 23 10:46 /var/crash/127.0.0.1-2011-08-23-10:46:22/vmcore

Read it (vmcore file), but had to download 2 packages:

[root@mytest ~]# ls kernel-uek-debuginfo-*
kernel-uek-debuginfo-2.6.32-100.35.1.el6uek.x86_64.rpm
kernel-uek-debuginfo-common-2.6.32-100.35.1.el6uek.x86_64.rpm

Because I didn't see kernel-uek-debuginfo-2.6.32-100.35.1.el6uek.x86_64* at http://oss.oracle.com/ol6/debuginfo

[root@mytest ~]# crash /usr/lib/debug/lib/modules/2.6.32-100.35.1.el6uek.x86_64/vmlinux /var/crash/127.0.0.1-2011-08-23-10\:46\:22/vmcore
crash 5.1.1-2.el6
Copyright (C) 2002-2011 Red Hat, Inc.
Copyright (C) 2004, 2005, 2006 IBM Corporation
Copyright (C) 1999-2006 Hewlett-Packard Co
Copyright (C) 2005, 2006 Fujitsu Limited
Copyright (C) 2006, 2007 VA Linux Systems Japan K.K.
Copyright (C) 2005 NEC Corporation
Copyright (C) 1999, 2002, 2007 Silicon Graphics, Inc.
Copyright (C) 1999, 2000, 2001, 2002 Mission Critical Linux, Inc.
This program is free software, covered by the GNU General Public License,
and you are welcome to change it and/or distribute copies of it under
certain conditions. Enter "help copying" to see the conditions.
This program has absolutely no warranty. Enter "help warranty" for details.

GNU gdb (GDB) 7.0
Copyright (C) 2009 Free Software Foundation, Inc.
License GPLv3+: GNU GPL version 3 or later <http://gnu.org/licenses/gpl.html>
This is free software: you are free to change and redistribute it.
There is NO WARRANTY, to the extent permitted by law. Type "show copying"
and "show warranty" for details.
This GDB was configured as "x86_64-unknown-linux-gnu"...

KERNEL: /usr/lib/debug/lib/modules/2.6.32-100.35.1.el6uek.x86_64/vmlinux
DUMPFILE: /var/crash/127.0.0.1-2011-08-23-10:46:22/vmcore [PARTIAL DUMP]
CPUS: 2
DATE: Tue Aug 23 10:45:06 2011
UPTIME: 00:05:17
LOAD AVERAGE: 0.06, 0.09, 0.04
TASKS: 109
NODENAME: mytest.surachartopun.com
RELEASE: 2.6.32-100.34.1.el6uek.x86_64
VERSION: #1 SMP Wed May 25 17:46:45 EDT 2011
MACHINE: x86_64 (1994 Mhz)
MEMORY: 1.9 GB
PANIC: "Oops: 0002 [#1] SMP " (check log for details)
PID: 1769
COMMAND: "bash"
TASK: ffff8800767982c0 [THREAD_INFO: ffff88007678c000]
CPU: 0
STATE: TASK_RUNNING (PANIC)

crash> log > log.txt
crash> exit
[root@mytest ~]# ls -ltr log.txt
-rw-r--r--. 1 root root 48084 Aug 23 10:51 log.txt

OK. I didn't write how to read it (vmcore file). I just posted "crashkernel" value, when you need to use KDUMP on Unbreakable Enterprise Kernel.

Related Post:
Just learn Kdump...