Jordan Savant # Software Engineer

Kernel Dev Blog 1 -- Setup


I want to grow, and there comes a point where you turn to yourself to do this. I started in calculator programming in grade school and then to web and graphics in high school. Web was a trajectory I was set on with my first job being a basic web developer contractor. It was forgiving and an excellent entry point into this career.

From there my education as an undergraduate in Software Engineering opened me up and improved me considerably from the hacker-diy coder to a principled engineer with foresight. I learned architectural principles, methodologies and a variety of ancillary fields of mathematics and business that helped make my skillset more valuables to others.

After college I did game development with a friend for years, working on the XBOX platform and PC platform mostly. This stretched me considerably as I had to develop for performance and scalability and put the Big^O principles into play. I learned about physics engines, data structures and algorithms such as pooling, quadtrees and pathfinding and even a considerable amount of packeted networking. Multi threaded development went from sandbox-theory to practice and proper object oriented development and memory management was essential for the code to survive our gaming dreams.

Now, back in the web world I have been stretched more in the dev ops and infrastructure areas over the last few years. CI pipelines, EC2 autoscalers, apache bench testing, DDOS and threat mitigations and scalable architectures of applications have been very important. AWS has become my bread and butter, becoming certified and wielding it like there is no other option.

But I find myself getting to that point of saturation, where the growth is beginning to diminish in my mid-thirties and I want to get better. Time to jump into a new pool of water. I truly respect those guys who now how to code for the machine, who speak its true language and provide incredible open source platforms for the world. Guys like Linus, Ken Thompson and John Carmack have always inspired me, so I figured I would take a stab at some lower level engineering: Linux, Device Drivers and Operating Systems in C.

This is my walk into that.

Using KernalNewbies to try and learn about Linux Kernel development:

1. Start

Installed VirtualBox with Ubuntu18 Server

Installed my ashes setup

Installed set of tools:

sudo apt-get install libncurses5-dev gcc make git exuberant-ctags libssl-dev bison flex libelf-dev bc

About the Tools

gcc is obviously the C compiler

make is the tool to build projecs and link files properly

git is obvious

libncurses5-dev lets us build terminal gui editors for menus etc

exuberant-ctags code index generator for editors

libssl-dev TLS and SSL protocols for key generation and signing etc

bison a "parser generator" for language grammars

flex lexical analyser generator for text pattern matching (combined with parser generator?)

libelf-dev Executable and Linkable Format standard file format for executable files, object code, libraries etc

bc math processing language

2. Download the Code

Cloned the repo on a specific branch

git clone -b staging-testing git://

Next is compiling, but reading first...

Digress:: Device Drivers CH1 Book

Driver Overview

Drivers provide an API to hardware without exposing the internals of the hardware

Separate "mechanism" from "policy", the drive should provide what can be done with the hardware now how it should be used

write kernel code to access the hardware, but don’t force particularpolicies on the user, since different users have different needs

Drivers are a software layer between applications and hardware and should take into consideration:

  • concurrent usage
  • synchronous and asynchronous operation
  • being opened more than once
  • less software layers to reduce "policy"
  • library for easier usage with applications

Kernel Overview

Kernel manages four main things:

  • Process Management
  • Memory Management
  • Filesystems
  • Device Control (device drivers embedded into the kernel for ever physical hardware attached)
  • Networking

Loadable Modules are a thing, meaning that object code can be loaded into the kernel at runtime to extend functionality

Object code Modules can be linked and unlinked with insmod and rmmod

Loading modules requires super user access and is a method that security vulnerabilities occur, root kits?a

Kernel can also be compiled without supporting modules and only drivers compiled with it function

Device and Module Classes

Char Devices (Character Devices) can be accessed as a stream of bytes

Block Devices host a filesystem for I/O larger than 512 bytes

Network Interfaces in charge of sending and receiving packets with other hosts


Security code is mostly policy and should be addressed at higher levels of code

You should however be conscious of dangerous operations that could damage systems or other users operations

Also coding against vulnerabilities such as buffer overflows are important

Other rules:

  • treat user input suspiciously
  • initialize all memory used to avoid random code existing in your runtime

Version Numbering

As far as the kernel is concerned, the even-numbered kernel versions (i.e., 2.6.x) are the stable ones for general distribution The odd versions (such as2.7.x), on the contrary, are development snapshots and are quite ephemeral

3. Copy Boot Configuration

The config for drivers etc is not included in the source (since its machine dependent) so we are gonna copy from Ubuntu

There is a config that lives in /boot that is thre kernel config file which is titled after the current release

Get current release with

$ uname -r # uname get system info, -r gets release version

Which pointed to a file /boot/config-4.15.0-91-generic

Copied to repo codebase

$ cp /boot/config-4.15.0-91-generic .config

Add a local version to the config file so you don't overwrite your OS release config


You can restore config values to default with make olddefconfig

You can use a cool ncurses editor with make menuconfig

4. Make and Compile

make or make -jX where "X" is the number of cores. (My vm was 2 core)

Ran out of HD space with so upped to 40GB

$ make -j2
  LD [M]  sound/usb/line6/snd-usb-pod.ko
  LD [M]  sound/usb/line6/snd-usb-podhd.ko
  LD [M]  sound/usb/line6/snd-usb-toneport.ko
  LD [M]  sound/usb/line6/snd-usb-variax.ko
  LD [M]  sound/usb/misc/snd-ua101.ko
  LD [M]  sound/usb/snd-usb-audio.ko
  LD [M]  sound/usb/snd-usbmidi-lib.ko
  LD [M]  sound/usb/usx2y/snd-usb-us122l.ko
  LD [M]  sound/usb/usx2y/snd-usb-usx2y.ko
  LD [M]  sound/x86/snd-hdmi-lpe-audio.ko
  LD [M]  virt/lib/irqbypass.ko


5. Install Kernel

Seems you can do it yourself and update GRUB config to launch into

Guide said Ubuntu has installkernel script already on it so we can use make to install

This script installs the kernel to /boot/, installs modules to /lib/modules/X.Y.Z/ (where X.Y.Z is something like 3.1.5), and updates file /boot/grub/grub.conf

This should put a new install into /boot

$ sudo make modules_install install
  INSTALL drivers/input/mouse/appletouch.ko
  INSTALL drivers/input/mouse/bcm5974.ko
  INSTALL drivers/input/mouse/cyapatp.ko
  DEPMOD  5.6.0-doomguyv1+
sh ./arch/x86/boot/ 5.6.0-doomguyv1+ arch/x86/boot/bzImage \ "/boot"
run-parts: executing /etc/kernel/postinst.d/apt-auto-removal 5.6.0-doomguyv1+ /boot/vmlinuz-5.6.0-doomguyv1+
run-parts: executing /etc/kernel/postinst.d/initramfs-tools 5.6.0-doomguyv1+ /boot/vmlinuz-5.6.0-doomguyv1+
update-initramfs: Generating /boot/initrd.img-5.6.0-doomguyv1+

run-parts: executing /etc/kernel/postinst.d/unattended-upgrades 5.6.0-doomguyv1+ /boot/vmlinuz-5.6.0-doomguyv1+
run-parts: executing /etc/kernel/postinst.d/update-notifier 5.6.0-doomguyv1+ /boot/vmlinuz-5.6.0-doomguyv1+
run-parts: executing /etc/kernel/postinst.d/x-grub-legacy-ec2 5.6.0-doomguyv1+ /boot/vmlinuz-5.6.0-doomguyv1+
Searching for GRUB installation directory ... found: /boot/grub
Searching for default file ... found: /boot/grub/default
Testing for an existing GRUB menu.lst file ... found: /boot/grub/menu.lst
Searching for splash image ... none found, skipping ...
Found kernel: /boot/vmlinuz-4.15.0-91-generic
Replacing config file /run/grub/menu.lst with new version
Found kernel: /boot/vmlinuz-5.6.0-doomguyv1+
Found kernel: /boot/vmlinuz-4.15.0-91-generic
Replacing config file /run/grub/menu.lst with new version
Updating /boot/grub/menu.lst ... done

run-parts: executing /etc/kernel/postinst.d/zz-update-grub 5.6.0-doomguyv1+ /boot/vmlinuz-5.6.0-doomguyv1+
Sourcing file `/etc/default/grub'
Generating grub configuration file ...
Found linux image: /boot/vmlinuz-5.6.0-doomguyv1+
Found initrd image: /boot/initrd.img-5.6.0-doomguyv1+
Found linux image: /boot/vmlinuz-4.15.0-91-generic
Found initrd image: /boot/initrd.img-4.15.0-91-generic

6. Boot to Kernel

We need to update the GRUB config file at /etc/default/grub


$ sudo update-grub2
Sourcing file `/etc/default/grub'
Generating grub configuration file ...
Found linux image: /boot/vmlinuz-5.6.0-doomguyv1+
Found initrd image: /boot/initrd.img-5.6.0-doomguyv1+
Found linux image: /boot/vmlinuz-4.15.0-91-generic
Found initrd image: /boot/initrd.img-4.15.0-91-generic
$ sudo reboot

From there the GRUB menu appeared and I chose "Advanced Ubuntu Options" where it showed the kernel version


Attempting to boot into my build resulted in a long error string that ended in

[     5.152144] ----[ end Kernel panic - not syncing: System deadlocked on memory ]---


A little bit of searching and I upped the memory size from 2GB to 4GB on the VM and it worked

$ uname -a
Linux doomkern 5.6.0-doomguyv1+ #2 SMP Sat Apr 4 22:23:48 UTC 2020 x86_64 x86_64 x86_64 GNU/Linux

7. GIT Email Setup

Now that we have the Kernel compiling and launching we need text based email system

We need to submit patches via GIT email which can be done with three functions:

sudo apt-get install esmtp a mail transport client

sudo apt-get install mutt a text based mail client that can send plan text patches

sudo apt-get install git-email to send email patches with git

8. Build CTAGS for Kernel Codebase

CTAGS is basically an indexer to find symbols in code (intellisense)

We already installed it so we can build the ctags with

$ make tags
  GEN     tags