Security Hardened Kernels For Linux Servers

Sunil Gadi

sunilgadi at gmail dot com
 
Table of Contents
1.  Introduction
2.  Pruning the kernel
3.  Kernel Logger
4.  Kernel Integrity Checker
5.  Read-Only File System
6.  Trusted Path Mapping
7.  Known Exploits
8.  Need for Proper Documentation
9.  Buffer Overflow
10. New Kernels
11. Patch for Latest Version of Kernel 2.6
12. Download
 
Introduction
 
This work is my Masters thesis "Hardened Kernels For Linux Servers" under the guidance of  Dr. Prabhaker Mateti, Associate Professor, Department of Computer Science and Engineering, Wright State University.  

 We believe that systems whose primary function is to be servers must deploy specially hardened kernels not only for performance reasons but even more importantly for security reasons.  We  build specialized kernels for Anonymous FTP server, Web server, Mail server, and a File server.  This thesis contributes a kernel source code patch for hardening kernels of Linux servers.
 

 Pruning The Kernel
 
 For a particular type of server, not all the services offered by the operating system are required during all the time.  A considerable amount of services are required only during the system boot up and for configuring the system.  There are some services which are not required during any period of time.  The attackers would exploit these unused services to compromise the system.  Our patch provides a finer control for system administrator to eliminate or restrict system calls, capabilities, memory devices, and ext file system attributes at compile time.  The patch also provides to freeze system calls, network interface configuration, and routing table configuration at runtime.
 
 During configuration of the kernel, we present a menu which shows all the system calls which are exploitable and not required, and system administrator could select among these system calls which should be eliminated at compile time.  We also added a new system call through which any system call, including itself, could be dynamically frozen while the system is up and running.  For the convenience of a system administration we have classified all the system calls of Linux kernel into various categories and subcategories so that, it would help while selecting or deselecting the system calls.
 
 Similar to system calls elimination, we present a menu through which the system administrator can select among the list of capabilities which should be eliminated at compile time.
 
 Kernel Logger
 
 There is no secure logging in standard Linux kernel.  All the kernel messages are written to a buffer which is read by a user process called klogd, through a system call, and are written to a file on the local system.  We designed and implemented a remote logging system at kernel level and this is done by a dedicated kernel thread called kernel logger.  The performance loss is almost negligible.  The IP address and port number of the remote log server is specified during kernel configuration.
 
 The kernel logger, we have designed, sends all the kernel messages to a remote system called log server. The kernel logger is started along with other kernel threads which are invoked by init kernel thread.  The kernel logger enters a continuous loop in which it check for logs in printk log buffer.  If the buffer is non-empty it makes a socket connection to the log server and sends the messages.  If the buffer is empty, it goes into a wait queue and sleeps until printk log buffer is non-empty.  
 
 If the connection to remote log server is not available then the kernel logger would relinquish  CPU voluntarily and joins the CPU run queue at the end.  The scheduling policy of kernel logger is set in such way that that it would not affect CPU allocation of other processes when connection to log server fails or when logs are flooded.
 
 During execution of shutdown  program, network is turned down long before the reboot system call is called.  So any kernel logs that are made after network is down are written to console.
 
   Kernel Integrity Checker
 
 To detect on-the-fly kernel modifications, we designed and implemented a new kernel integrity checking system which would check the integrity of text of kernel using crypto algorithms introduced recently into the the kernel. This job is done by a kernel thread called Kernel Integrity Checker.
 
 
 When KIC is started, it first computes the MD5 checksum of each page of the kernel text region.  The addresses of the pages, their respective MD5 checksums are stored in MD5 database which is also inside the kernel's memory. Once the MD5 checksums are stored in the MD5 database KIC goes to sleep. During this period, from the moment kernel thread is started till it goes to sleep for the first time, there will be no context switch.  KIC wakes up periodically at regular intervals to compute the MD5sums of the running kernel.  If the MD5 checksum comes out to be different from that in the database it writes to the printk buffer which is read by the kernel logger.  The message that is written to the printk buffer will contain the addresses of the page which is modified.  This will give enough details for the administrator to analyze the attack.  Then it is left to the discretion of the system administrator to take the next action.
 
 
 During the period, from the moment KIC wakes up from the sleep till it goes to sleep again after doing integrity check, there will be no context switch.  The length of the interval at which KIC wakes up periodically is reasonably adjusted to see no performance loss is suffered by other processes.  This can be configured by system administrator during kernel compilation.  KIC exits when the init process or shutdown program calls reboot system call. The init process informs KIC about the system shutdown before the it shutdown the power.
 
We can extent this idea to detect modifications done to data structures which are not expected to change such as system call table and IDT table.
 
 Limitations
 By modifying the MD5 checksums stored in database, the attacker can generate all kinds of false alarms for the administrator.  He can also modify the kernel and replace the MD5 checksum in the database with the corresponding new checksum  so that KIC cannot detect the modification.  However it makes an attacker's attempts more harder.
 
   Read-Only File System
 
 Attacker with root privileges can have access to any file on the system.  He can also access raw devices and corrupt the file system on it.  We designed a read-only file system which is based on intercepting system calls and this would also prevent modifying file system through raw devices.  The administrator has to select which files should be allowed for modification for the functionality of servers.
 
   Trusted Path Mapping
 
 Trusted Path Execution (TPE) prevents file execution from arbitrary paths.  TPE restricts file execution to some specific directories and those directories are called trusted directories.  The TPE of Grsecurity, considers all the root owned directories, which are not others writable, as trusted directories.  We improved this in a way that administrator can specify a list of trusted directories and even superuser cannot violate.  These trusted directories are specified while configuring the kernel.
 
 
 We have designed and developed ``Trusted Path Mapping'' (TPM) in  which any file which has to be mapped to process address space should be in one of the trusted path directories. The system calls which are intercepted are execve, mmap, mprotect.
 
 TPM consists of TPM monitor and TPM directory database.  Any call to execve, mmap, or mprotect should pass through the TPM monitor.  The monitor would search whether the file which is passed as an argument to the system call is in any of the trusted path directories stored in TPM directory database.  If the file is not in the trusted directories, the system call is denied.  TPM directory database contains i-node numbers and device numbers of the trusted directories.  During kernel compilation, system administrator has to specify the list of trusted directories separated by commas.
 
 TPM can be activated in two different ways.  The first one is through the init kernel thread.  The init kernel thread, before entering user mode, would lookup the file system for the i-nodes of the directories and fills the TPM directory database.  No other process would modify the database.  If one of trusted directory is in a file system which is not mounted before init kernel thread become a user process, then i-node lookup fails.  For this, a second way of activating TPM is introduced.
 
 The second way of activating TPM is through a newly introduced system call called tpm.  Once all the file systems are mounted this system call is called which does i-node lookup and fills TPM directory database.
 
 We recommend the first way because in case of the second one there is longer window for the attacker.
 
 
 Known Exploits
 
 In building a new hardened kernel, we wish to have, at a minimum, a guarantee that old exploit techniques do not work any more.  For this we added some of the already existing secure patches to our patch. We adopted Grsecurity's chroot security features into our patch. We adopted OWL's temporary file race condition prevention feature into our patch. Irrespective of whether close-on-exec flag is set on a file descriptor, our patched kernel closes all the files open before execve.  Because of this some programs which expect files to be open through execve may crash. The other prevention techniques we adopted into our kernel are LIDS protecting important process feature, IP tables freezing etc.
 
 Need for Proper Documentation
 
 Secure patches released by open source hackers have no proper documentation.  There is no proper explanation of about how the patch prevents the exploits, what are various limitations of the patches, side effects of these patches, and what is the root cause of the exploitation.
 
 We documented the design and implementation details of all the features of our patch.  For ``pruning the kernel'', we have explained , in details, various exploitable services which are not required for the functionality of servers.
 
 We also explained, with examples, the root causes for various types of known attacks including LKM based rootkits, /dev/kmem rootkits, buffer overflow, temporary file race condition, proc file system exploits, chroot jail breaking, and ptrace exploits, and suggestions are made to prevent them.  We analyzed various secure patches available which aim to prevent these exploits and exposed the limitations of these patches.
 
   Buffer Overflow
 
 We reviewed five independent modifications, known as OWL, KNOX, paged-PAX, segmented-PAX patches, and RSX module, made to the Linux kernel that aim to prevent buffer overflow exploits in IA-32 based Linux.  We showed that the OWL and RSX patches are ineffective, even though their ideas are workable.  We brought attention to the fact that Linux on IA-32 does not use segmentation wisely.  We also discussed the performance impact on the kernel and benchmarked the performance of patched kernels using lmbench.
 
   New Kernels
 
 We provided detailed explanation of various secure kernel features which should be set for a specific type of server.  Finally We built specialized kernels for web server, FTP server, mail server, and file server.
 
 The kernels we built are version 2.4.23 and are tested with Linux Mandrake Distribution 9.2 running over Intel Pentium III processor.  Though we developed our patch for kernels on IA-32 architecture, most of the features, being architecture independent, are expected to work on other architectures too.  But we never tested them on other architectures.
 

 Patch for Latest Version of Kernel 2.6
 I am currently working on porting all the work to latest version in 2.4 and 2.6.                                             


 Download
 All the files including the Linux kernel source patch, thesis documentation and slides are downloadable at  project page at Sourceforge