Linux Kernel Reference

Linux Kernel Reference

--> How Do I Load a Kernel Module on System Start-up?

On startup, a RHEL system will automatically check for the file /etc/rc.modules.
This file should contain additional modules that require loading on startup.
The following command can be used to add a module to this file and ensure that it is loaded on the next system startup:
# echo “modprobe <module_name>” >> /etc/rc.modules && chmod 755 /etc/rc.modules

--> What is the technical differences between a normal SMP kernel and a hugemem kernel?

The original i686 only had 32-bits with which to address the memory in a machine. Because 2^32 == 4GB, the original i686 can only address up to 4GB of memory.

Sometime later on, Intel introduced the PAE extensions for i686. This means that although the processor still only runs 32-bit code, it can use up to 36-bits to address memory (they added more bits later on, but that’s not super-relevant). So newer i686 processor could address 2^36 bits of memory, or 64GB.

However, the linux kernel had limitations that makes it essentially impossible to run with 64GB of memory reliably Therefore, while running the linux kernel on an i686 PAE machine, you can really only use up to about 16GB of memory reliably.

To work around that limitation, RHEL had a new mode called “hugemem”. Among other things, it allowed the kernel to reliably address all 64GB of memory.

Hugemem variant allowed you to access up to 64GB of memory “among other things”.The other thing it allowed, in particular, is for userspace to address 4G of memory per process, instead of the usual 3G. This is another benefit that some customers found useful.

--> How to specify a ‘default’ boot kernel type for a RHEL?

The kernel type which should be the default is determined by the configuration file /etc/sysconfig/kernel, the system default defined in this file will cause the latest kernel to be configured for booting by default.

--> How to update RHEL kernel ?

Actually, Redhat won’t recommends to add a new kernel to the system instead of updating, which mean we can choose to run the system on old kernel or the new one, so that we can rollback to old kernel easily.

To install new kernel package:
Step1 : Get the kernel package, you can download the kernel package from Red Hat Network (RHN), please confirm which type of kernel you need!
In RHEL4 you can choose a normal kernel, kernel-smp, kernel-largesmp, kernel-hugemem.
In RHEL5, you can choose a normal kernel, kernel-PAE and kernel-xen.
Step2 : Install kernel package
Note: Again, never use “rpm -Uvh” here.
# rpm -ivh kernel-<version>.rpm
If you have registered your system to RHN, and your system is Red Hat Enterprise Linux 5, run the command:
# yum install kernel
If you have registered your system to RHN, and your system is Red Hat Enterprise Linux 3 or 4, run the command:
# up2date kernel
Step3: If your system is IA32 or x86_64 architecture, edit the /boot/grub/grub.conf. You will find a new kernel section is appended to this file after you install the new kernel. Find the “default” field, make sure it points to the right
kernel you want to boot. You can modify the value of “default”, just remember this value is counted from “0″ which means the “0″ is the first kernel, the “1″ is the second kernel.
If your old system has installed some third party drivers, these drivers will not be included in new kernel, you need to reinstall all the third party drivers and corresponding initrd.
Normally, only the new kernel package is needed, but sometimes the new kernel need install or upgrade other packages first (dependency problem), such as kernel-devel, kernel-headers, ecryptfs-utils, mkinitrd, etc.
If your old system has installed kernel-devel and kernel-headers package, and if you want to keep the kernel-devel package consistent with the new kernel, you can update the kernel-devel with command “rpm -Uvh”, just remember that after you update it, it might not be easy to get it back to the old version, and some of your development applications might be affected.



--> How do I exclude updating the kernel or other packages in RHEL 5/6 while updating system via yum ?
The up2date command in RHEL 4 excludes kernel updates by default. The yum in RHEL 5 includes kernel updates by default.
To skip installing or updating kernel or other packages while using the yum update utility in RHEL 5 and 6, use following options
Via command line (temporary):
# yum update –exclude=PACKAGENAME
For example, to exclude all kernel packages:
# yum update –exclude=kernel*
To make permanent changes, edit the /etc/yum.conf file. Follow these steps:
# vi /etc/yum.conf
Add a new line in the [main] section like this: exclude=package1,package2
For example:
…
gpgcheck=1
plugins=1
installonly_limit=3
exclude=kernel*
Save file.
After the file is saved, run yum update. The packages specified should not install or update.



--> How do I boot using an alternate/previous kernel?
As the system is booting, we see a message similar to the following:
Press any key to enter the menu
Booting Red Hat Enterprise Linux (2.6.32-279.el6.x86_64) in 3 seconds…
At this stage, if you press/send any key, you will be placed into the GNU Grub menu. Here you will be able to use the arrow keys to select and boot into any kernels that are installed on your system. Highlight the previous kernel and press/send Enter to boot using that kernel.

How do I modify GRUB (grub.conf) to use a new kernel?

In order for the system to boot to a new kernel, the boot loader needs to have an entry with the new kernel. If using GRUB, the configuration file is /etc/grub.conf. Here is an example of the /grub.conf file:
# grub.conf generated by anaconda
#
# Note that you do not have to rerun grub after making changes to this file
# NOTICE: You have a /boot partition. This means that
# all kernel and initrd paths are relative to /boot/, eg.
# root (hd0,0)
# kernel /vmlinuz-version ro root=/dev/hda7
# initrd /initrd-version.img
#boot=/dev/hda
default=0
timeout=5
splashimage=(hd0,0)/grub/splash.xpm.gz
hiddenmenu
title Red Hat Enterprise Linux AS (2.4.21-18.EL)
root (hd0,0)
kernel /vmlinuz-2.4.21-18.EL ro root=LABEL=/
initrd /initrd-2.4.21-18.EL.img
Add an entry for the 2.4.21-20 kernel. In order to do this, there are several changes to be made. First, add the appropriate stanzas defining the root, kernel image, initrd and any additional options. The easiest way to do this is to copy the lines from root=, kernel and initrd and just change the file names to the new kernel.
# grub.conf generated by anaconda
#
# Note that you do not have to rerun grub after making changes to this file
# NOTICE: You have a /boot partition. This means that
# all kernel and initrd paths are relative to /boot/, eg.
# root (hd0,0)
# kernel /vmlinuz-version ro root=/dev/hda7
# initrd /initrd-version.img
#boot=/dev/hda
default=0
timeout=5
splashimage=(hd0,0)/grub/splash.xpm.gz
hiddenmenu
title Red Hat Enterprise Linux AS (2.4.21-18.EL)
root (hd0,0)
kernel /vmlinuz-2.4.21-18.EL ro root=LABEL=/
initrd /initrd-2.4.21-18.EL.img
title Red Hat Enterprise Linux AS (2.4.21-20.EL)
root (hd0,0)
kernel /vmlinuz-2.4.21-20.EL ro root=LABEL=/
initrd /initrd-2.4.21-20.EL.img
Now that the grub.conf file has the updated kernel information, boot into the new kernel by default by changing the default= option. The first stanza is 0, second stanza is 1, etc. To boot by default into the new kernel specify default=1.
# grub.conf generated by anaconda
#
# Note that you do not have to rerun grub after making changes to this file
# NOTICE: You have a /boot partition. This means that
# all kernel and initrd paths are relative to /boot/, eg.
# root (hd0,0)
# kernel /vmlinuz-version ro root=/dev/hda7
# initrd /initrd-version.img
#boot=/dev/hda
default=1
timeout=5
splashimage=(hd0,0)/grub/splash.xpm.gz
hiddenmenu
title Red Hat Enterprise Linux AS (2.4.21-18.EL)
root (hd0,0)
kernel /vmlinuz-2.4.21-18.EL ro root=LABEL=/
initrd /initrd-2.4.21-18.EL.img
title Red Hat Enterprise Linux AS (2.4.21-20.EL)
root (hd0,0)
kernel /vmlinuz-2.4.21-20.EL ro root=LABEL=/
initrd /initrd-2.4.21-20.EL.img
Save the changes made and the next time the machine boots, it will boot into the new kernel by default.
Kernels supplied by Red Hat will add an entry automatically to the installed bootloader. It is still possible to modify the /etc/grub.conf file to specify which kernel to load automatically.

--> How to downgrade a kernel?

Install the required kernel using it’s complete version number.

#yum install “kernel-2.6.18-194.8.1.el5″

Once installed boot the machine in that kernel.

--> How to set kernel parameters in sysctl.conf file permanently?

Edit /etc/sysctl.conf file.
# vi /etc/sysctl.conf
Add Required entries in sysctl.conf, as given belo.
kernel.sem=1000 32000 32 512
kernel.msgmni = 1024
fs.file-max = 65535
kernel.shmmax = 2147483648
net.ipv4.tcp_max_syn_backlog = 8192
Note : Make sure same parameter entries are not duplicated in the file /etc/sysctl.conf.
Run sysctl with -p parameter to load in sysctl settings from the default file /etc/sysctl.conf.
# sysctl -p
Note : Before modifying the values of above kernel parameters please check their limits and after effects on the system.

--> What is kernel-firmware?

Kernel Firmware includes all the necessary files required by firmware for some devices to operate (interacting with the kernel)

Name : kernel-firmware
Arch : noarch
Version : 2.6.32
Release : 131.12.1.el6
Size : 2.5 M
Repo : production-rhel-x86_64-workstation-6
Summary : Firmware files used by the Linux kernel
License : GPL+ and GPLv2+ and MIT and Redistributable, no modification per
Description : Kernel-firmware includes firmware files required for some devices to operate.

--> How to increase the kernel ring buffer size?
To avoid losing boot messages, the kernel log buffer must be a large enough size to contain all the boot messages up until the point the system is able to write the log to permanent storage. If the kernel log buffer is not large enough, the oldest messages during boot will be lost, overwritten by the latest messages.
By default, this is set in the conf variable : CONFIG_LOG_BUF_SHIFT, to check :
# grep CONFIG_LOG_BUF_SHIFT /boot/config-2.6.18-238.5.1.el5
CONFIG_LOG_BUF_SHIFT=19
Here 19 means the the bits have to be shifted by 19, which is :
2 ^ 19 = 512 KB
So, the ring buffer is set to 512 Kilo Bytes
This variable is set during compilation, and cannot be changed, unless you are recompiling the kernel (which is not supported)
During runtime the buffer size may be increased with the log_buf_len kernel parameter:
log_buf_len=n Sets the size of the printk ring buffer, in bytes.
Format: { n | nk | nM }
n must be a power of two. The default size is set in the kernel config file.
A setting of ‘log_buf_len=3M’ will be enough to register a huge amount of boot messages.
To set a larger kernel log buffer of 3MB, add ‘log_buf_len=3M’ to the kernel parameters in the /boot/grub/grub.conf file.



--> How do I blacklist a kernel module to prevent it loading automatically?
In order to prevent kernel modules loading during boot, the module name must be added into the blacklist file.
Ensure the module is not configured to load either in /etc/modprobe.conf, /etc/modprobe.d/*, /etc/rc.modules, or /etc/sysconfig/modules/* the run the following commands:
Red Hat Enterprise Linux 5 and 6
To prevent a module from being loaded directly you can use the command:
echo “blacklist <module name>” >> /etc/modprobe.d/blacklist.conf
This will not prevent a module being loaded if it is a required or optional dependency of another module. Some kernel modules will attempt to load optional modules on demand.
This can be achieved by running the following command:
echo “install <module_name> /bin/false” >> /etc/modprobe.d/blacklist.conf
This will take effect the next time that the module is attempted to load. There may be unexpected side affects if a module is blacklisted that is required for other specific hardware.
Red Hat Enterprise Linux 4
echo “alias <module-name> off” >> /etc/modprobe.conf
Remove Module Temporarily
It is possible to remove any currently-loaded module by running:
modprobe -r <module name>



--> What is the proper method to remove old kernels from a Red Hat Enterprise Linux system?
Kernels, like all other files on Red Hat Enterprise Linux, if installed via a supported method, are part of an RPM package. To show your currently installed kernels issue the following command:
rpm -q <package_name>
Tip: <package_name> would be the name of the RPM package. For example, kernel. For system using other kernel varietals, like SMP, hugemem, or enterprise, the package names would be kernel-smp, kernel-hugemem, and kernel-enterprise, respectively.
Choose which kernel you want to uninstall from the list of those installed. Then use the following command to remove the kernel package:
rpm -e <package_name>
Warning: Do not remove the kernel the system is currently running. To verify which kernel the system currently has loaded, use the following command:
uname -a
For example:
#uname -a
Linux tavarua 2.4.21-27.0.2.EL #1 Wed Jan 12 23:46:37 EST 2005 i686 i686 i386 GNU/Linux
# rpm -q kernel
kernel-2.4.21-4.EL
kernel-2.4.21-15.0.3.EL
kernel-2.4.21-15.0.4.EL
kernel-2.4.21-27.0.2.EL
# rpm -e kernel-2.4.21-4.EL
In the above example, the system was using the 2.4.21-27.0.2.EL kernel and the oldest kernel on the system, 2.4.21-4.EL, was removed using the rpm command.

Note: Red Hat Network (RHN) can download and install new kernels for a registered system. However, it will leave the old kernels in place. You can manually remove them if you choose, although it is a good idea to keep at least two kernels installed: the most current one and at least one older version that is known to work for your system.

--> What are the kernel parameter for inter-process communication?

There are three types of inter-process communication: Shared Memory, Semaphore, and Message Queue, there are some kernel parameters could be tuned.
By far our kernel-doc package has no description of these parameter, but you can refer to “man proc” for more information.
kernel parameter –> kernel variable description
Shared Memory
/proc/sys/kernel/shmmax SHMMAX –> the max allowable size of one shared memory segment
/proc/sys/kernel/shmmni SHMMNI –> the max number of shared memory segments in the entire system
/proc/sys/kernel/shmall SHMALL –> the max size of shared memory in page (one page = 4k bytes for x86/x86-64)
Semaphore
/proc/sys/kernel/sem SEMMSL –> the max number of semaphores per identifier
/proc/sys/kernel/sem SEMMNS –> the max value of a semaphore.
/proc/sys/kernel/sem SEMOPM –> the max number of operations per semaphore call
/proc/sys/kernel/sem SEMMNI –> the max number of semaphore sets in the entire system
Message Queue
/proc/sys/kernel/msgmni MSGMNI the number of message queue identifiers
/proc/sys/kernel/msgmnb MSGMNB the default max size of a message queue
/proc/sys/kernel/msgmax MSGMAX the max size of message (bytes)



--> Root cause analysis for kernel panic on RHEL?

A vmcore contains relevant diagnostic information from the contents of memory at the time of the panic
For RHEL 4:

Configure a netdump client and netdump server to gather a virtual memory core (vmcore)

The steps to configure a netdump server and netdump client can be found in How do I configure netdump on Red Hat Enterprise Linux 3 and 4?

For RHEL 5 & 6:

Configure kdump and/or server to receive/hold the vmcore files

The steps to configure kdump can be found in kexec/kdump on Red Hat Enterprise Linux?

--> How can the runtime values of kernel module parameters be displayed in RHEL5?
The systool command can be used to query kernel module information:
# systool -avm <module_name>
For example, to display the current values being used for the scsi_mod parameters:
# systool -avm scsi_mod
Module = “scsi_mod”
…
Parameters:
default_dev_flags = “0″
inq_timeout = “5″
max_luns = “512″
max_report_luns = “511″
scsi_logging_level = “0″



--> The /boot directory is getting full, how do I remove old kernels?

The default behavior of up2date and yum is to install new versions of the kernel, rather than to upgrade the existing kernel. This results in multiple installed kernel packages on the machine. This is done in order to allow you to boot the system with a previous version of the kernel should a new kernel prove to be incompatible with some aspect of your system configuration. Over time your system will accumulate older kernel versions that may no longer be necessary and will take up space. These kernels may be removed if they are no longer needed.

When removing older versions of the kernel be very careful to remove only unneeded kernel versions and to not mistakenly uninstall kernel versions that you need for your system, including the currently running kernel. If you want to remove the currently running kernel, reboot the system using another kernel and then perform the uninstall.

# df
Filesystem 1K-blocks Used Available Use% Mounted on
/dev/mapper/VolGroup00-LogVol00
14804452 7006156 7046272 50% /
/dev/sda1 101086 101086 0 100% /boot
none 127332 0 127332 0% /dev/shm

# rpm -qa | grep kernel-unsupported
kernel-unsupported-2.4.21-15.0.3.EL
kernel-unsupported-2.4.21-15.0.2.EL

# rpm -e kernel-unsupported-2.4.21-15.0.2.EL

# rpm -qa | grep kernel-unsupported
kernel-unsupported-2.4.21-15.0.3.EL

--> What are the kernel module utilities in Red Hat Enterprise Linux?

A group of commands for managing kernel modules is available in Red Hat Enterprise Linux. Use these commands to determine if a module has been loaded successfully or when trying different modules for a piece of new hardware.

To list currently loaded modules, use lsmod command:

# /sbin/lsmod
Module Size Used by Not tainted
iptable_filter 2412 0 (autoclean) (unused)
ip_tables 15864 1 [iptable_filter]
nfs 84632 1 (autoclean)
lockd 59536 1 (autoclean) [nfs]
sunrpc 87452 1 (autoclean) [nfs lockd]
soundcore 7044 0 (autoclean)
ide-cd 35836 0 (autoclean)
cdrom 34144 0 (autoclean) [ide-cd]
parport_pc 19204 1 (autoclean)
lp 9188 0 (autoclean)
parport 39072 1 (autoclean) [parport_pc lp]
autofs 13692 0 (autoclean) (unused)
e100 62148 1
microcode 5184 0 (autoclean)
keybdev 2976 0 (unused)
mousedev 5656 1
hid 22308 0 (unused)
input 6208 0 [keybdev mousedev hid]
usb-uhci 27468 0 (unused)
usbcore 82752 1 [hid usb-uhci]
ext3 91464 2
jbd 56336 2 [ext3]

For each line, the first column is the name of the module, the second column is the size of the module, and the third column is the use count.

The information after the use count varies slightly per module. If (unused) is listed on the line for the module, the module is currently not being used. If (autoclean) is on the line for the module, the module can be autocleaned by the rmmod -a command. When this command is executed, any modules that are tagged with autoclean, that have not been used since the previous autoclean action, are unloaded.

Note : Red Hat Enterprise Linux does not perform this autoclean action by default.

If a module name is listed at the end of the line in brackets, the module in the brackets is dependent on the module listed in the first column of the line. For example, in the line

usbcore 82752 1 [hid usb-uhci]
the hid and usb-uhci kernel modules depend on the usbcore module.

The /sbin/lsmod output is the same as the output from viewing /proc/modules.

To load a kernel module, use the modprobe command followed by the kernel module name.

By default, modprobe attempts to load the module from the /lib/modules/<kernel-version>/kernel/drivers/ subdirectories. There is a subdirectory for each type of module, such as the net/ subdirectory for network interface drivers. Some kernel modules have module dependencies, meaning that other modules must be loaded first for it to load. The /sbin/modprobe command checks for these dependencies and loads the module dependencies before loading the specified module.

For example, the following command will load any module dependencies and then the hid module:

# /sbin/modprobe hid
To print to the screen all commands as /sbin/modprobe executes them, use the -v option. For example:

# /sbin/modprobe -v hid

/sbin/insmod /lib/modules/2.4.21-1.1931.2.399.ent/kernel/drivers/usb/hid.o
Using /lib/modules/2.4.21-1.1931.2.399.ent/kernel/drivers/usb/hid.o
Symbol version prefix ‘smp_’
The /sbin/insmod command also exists to load kernel modules; however, it does not resolve dependencies. Thus, it is recommended that the /sbin/modprobe command be used.

To unload kernel modules, use the /sbin/rmmod command followed by the module name.

The rmmod utility only unloads modules that are not in use and that are not a dependency of other modules in use.

For example, the command below will unload the hid kernel module.

# /sbin/rmmod hid
To see the detail information about kernel modules, use the /sbin/modinfo command.

Another useful kernel module utility is modinfo. It displays information about a kernel module. The general syntax is:

# /sbin/modinfo [options] <module>
Options include -d which displays a brief description of the module and -p