The Zana Zen

Emulators in computer science are computer software that emulates an environment for another software to run, it may include some hardware devices as well, but if a complete environment is emulated that it will be usually called a virtual machine. An example for such enironment is DOSEMU, a DOS operating system environment which is emulated and designed to run on top of Linux operating system. The core for any software emulator is the CPU emulation part, many virtual machines uses the CPU that the bochs project had created, it can run x86, AMD64, and PPC software.

I played a bit with emulators in the past, and I created a simple python CPU emulator that emulates “flat” assembly code that can be defined by a configuration file. In this post I’ll describe its features, how does it work and what can be achieved by using it.

Simple Python CPU Emulator

The CPU emulator itself has a couple of runtime variables that are being used by the parsers, it moves line by line over the assembly that you tell it to emulate, and if the current assembly line matches a pattern (defined in the configuration line) then it evaluates another pattern, according to the run-time state of the CPU registers.

The configuration file of the emulator has two parts, it defines the registers that the cpu holds and the instructions that the emulated CPU can execute. The design is very simple and elegant, when the emulator initializes it prepares an opcode python dictionary. For instance, the opcode INC is defined as follows in the configuration file:

INC $reg1 | $reg1 += 1

When the parser encounters that configuration line, it adds the following key => value to the dictionary:

opcodes["INC (.*?)"] = “cpu[execed.group(1)] += 1″

Where cpu[execed.group(1)] is the run-time CPU register that is matched by the CPU emulator parser. When the parser reads a new assembly line, it first moves through all the keys of the opcodes dictionary, and tries to do a regular expression match. If it finds one, then the value of the dictionary is being simply evaluated. Here is the source code of the emulation loop:

01: while (cur_op_line < end_of_ops):
02: opcode = a_ops[cur_op_line]
03: execed, ptrn = get_exec(opcode)
04: if (execed):
05: if a_verbosity: print "%d:\t%s" % (cur_op_line+1, opcode)
06: exec(opcodes[ptrn])
07: else:
08: print "[-] havn't found a match for", opcode
09:
10: cur_op_line += 1

A Riddle For Fun

My friend Imri gave me this riddle when he took a course in complexity at the Tel Aviv University.

You a have a CPU with 4 registers (r0 through r3) and 3 basic opcodes that works as described:

• INC reg – increases the value of reg by 1
• DEC reg – decreases the value of reg by 1
• TEST reg, line – if reg != 0 then start executing opcodes at the specified line

Download the source code, it comes ready with the configuration file for this exercise, and write a program that will end the execution when one of the registers contains the value 144.

• The CPU starts when all the registers are initialized to zero.
• The program length must not exceed 20 instructions.

Have pun!

Abstract

This article is kind of a continuation for my under-the-hood article series, you can take a look at my previous articles regarding the PCI bus. In this article, I delve further deeper into interrupts, and we’ll explore what exactly happens when an interrupts occurs.

If you are a software developer, this information will help you grasp the PC architecture a little bit better. This article’s intent is to encourage people to explore things and widen their knowledge… I see too many people which are trapped inside their operating system’s environment, especially if they are using the M$ black box. This is the only reason I see for switching to Linux Ok, so let’s start!

Interrupts

An interrupt is basically a signal from the hardware that tells the software to perform an operation. It is handled by the operating system that calls the ISR (Interrupt Service Routine) for that interrupt request (IRQ).

Generally, we distinguish between two different types of interrupts, Edge-Triggered and Level-Triggered. Edge-Triggered interrupts are interrupts which are being caused by changing the bus line level, it is basically a transition from a 1 to 0 or from 0 to 1 (falling-edge and rising-edge repectively). This “old fashion” type of interrupt was used in the ISA bus. The problem with this type of interrupt is that it is difficult to be shared, that means, several devices couldn’t shared the same IRQ line. Level-Triggered interrupts are being caused by raising or lowering the level of the bus line and holding it right there until the interrupt is serviced, they were used in the original PCI bus (and are still being used) as the standard type of interrupt.

Interrupt Controllers

The interrupt controller’s goal is to provide interrupt capabilities to the main processor (CPU) through a single line, when a device issues an interrupt, it is delivered to one of the interrupt controller’s IRQs (pins), from there the interrupt is generated in the CPU, which, in turn, checks with the interrupt controller for the source of the interrupt through a special register which are being hold at and managed by the interrupt controller. Old interrupt controllers, such as the PIC (Programmable Interrupt Controller) provided interrupt-priority, interrupt-masking and general flexibility for dealing with interrupts in the platform. Old devices where programmed to use fixed IRQs and problems arose when two or more devices shared the same IRQ, if the PIC was programmed to be used in the edge-triggered mode, then serious conflicts could cause the system to hang or not function at all. Well… In the edge-triggered mode interrupt actually could be shared if the devices were specially built for this event. But since the operating system must run all the ISRs that exists in the chain that is associated with that specific IRQ that is signaled, it is not so effective after all.

In the Level triggered mode, the PIC knew how to share IRQs between different devices, but sharing interrupts is not a good deal in any case, this of course leads to performance issues and faults that are being caused by poorly written device drivers.

Consider the following scenario: Device A, which shares his IRQ with Device B, signals its driver, the interrupt is issued and the operating system processes it. The chain of ISRs contains two different ISRs. First the driver for Device B is processing the interrupt becuase he is first in the chain, and actually decides that he is going to handle the interrupt (because its poorly written ISR handles any interrupt). The operating system sees that the interrupt was handled and stops executing the ISR-chain and acknowledges the PIC. After some time Device A sees that the interrupt wasn’t handled by its driver and issues the same interrupt again. This scenario leads to interrupt storms or causes Device A to stop function.

To solve this problem and allow more flexibility, Advanced Programmable Interrupt Controllers (APICs) were introduced, they contain more IRQs and their function is better adapted to the operating system. Windows, for example uses its IRQL mechanism to mask interrupts in the APIC, this is accomplished by a single mov assembly instruction, instead of runing several I/O port instructions (such as in or out) in the case of using a PIC.

APICs are being used in Multiprocessor environments, each CPU has its Local APIC and another controller, called I/O APIC actually routes interrupts from the bus to the LAPIC. This allows greater flexibility, and sharing interrupts is no longer an issue, since each LAPIC has 24 IRQs. Another advantage is that each APIC has it’s own timer, this allows each CPU to better schedule the CPU time distributed between threads in quantum units. APICs are also supports IPIs (Inter-Processor-Interrupt), a way of one processor to interrupt another processor. IPIs are being used for synchronization and cache-coherency. The I/O APIC’s function is to distribute interrupts between the CPUs in a multi-processor environment.

Final Words

If you bared with me to this point, It is surely admirable! even I couldn’t bare with myself writing this post

Anyway, we saw how interrupts are being generated, how they are routed at some platforms which are based on the PIC and APIC implementation and we also took a look at how the operating system handles interrups. I hope this post was enjoyable for you.

Evilbitz.

This is the second part of my introduction to PCI article. If you haven’t read the first part, it is highly recommended that you’ll read it first.

So, in part I, we learned that all our PCI devices are connected through this main bus line to the CPU and memory. Any entity in the system can use the PCI bus, but it will first ask the bus arbiter to do so. A bus master is any entity that holds the bus and has an ability to write to it.

Any device can hold memory resources that are being used to transfer data between the driver and the device. Those addresses may be used for storing the current display buffer for example. This memory resource is part of the main computer’s RAM, so if the driver will want to copy data to the buffer, it will just copy the data to the RAM, to the address where the BIOS mapped the resource to.

BIOS stands for Basic Input and Output System, it is reliable to boot the system and provide it with some basic capabilities like primitive display adapter support. One part of the bios, moves through the existing PCI devices and supplies them with the memory resources that they require. After doing so, it boots the operating system which enumerates the PCI devices and existing memory resources.

Each PCI device is identified by a device ID and a vendor ID, both are part of the standard 64 bytes configuration space that each PCI device holds. So after the operating systems probes for this info, it searches for an appropriate driver to handle this device, this is being done by matching the device/vendor IDs.

In windows there is a kernel module which is called the Plug and Play manager that loads the right driver and let it initialize itself. There is also a file, for each driver, that describes these IDs. These files are called INF files and they provide info for the PnP manager for loading and installing the appropriate driver. To look at some examples, you can dig in the %windir%\inf directory.

Let’s go back to the memory resources. As I said, these are just some memory regions from the computer’s RAM that are being mapped to the PCI device as well. When a driver needs to write data it may utilize the main CPU for this task or it may use a special hardware unit which is called DMAC.

DMAC stands for Direct Memory Access Controller and it’s basically a processor that knows how to copy data from one place to another. Drivers are using this controller to carry out copying operations thus letting the main CPU be free to carry out other tasks.

DMA may be operate in two ways, the standard one, which by the DMAC reads each time the following byte from the source (the PCI device for example) and copy it to it’s destination (the RAM).

More sophisticated DMACs are using a method which is called flow-through. By which the controller just playing with signals over the bus and holds no actual data by itself. It operates in this manner: on the same time it issues a read operation to the device and a write operation to memory, letting the signals flow smoothly over the bus, the DMAC will keep doing this until it will finish its transaction.

I hope that this series of articles where informative and enjoyable for you. Keep to check out for new posts on this cool blog!

This article is written in simple language and is intended for the common knowledge of the average computer user that should know what is happening under the “hood”, yet most advanced users can find this information interesting and it might introduce some new topics for them.

Before we will delve into the interesting part, let’s move through some basics. PCI is an acronym which stands for Peripheral Component Interconnect. It is the medium by which the CPU is talking with the rest of the devices in the computer. In the computer Jargon, it is called a bus. It is a set of lines that connects all the devices to the CPU and RAM (Random Access Memory) all together. For example, a sound card may be connected into a PCI slot on the Mother Board. When the operating system boots, it probes for PCI devices and get’s a list of all the devices which are connected in the system.

Each device owns the bus for just a period of time. Since there are several devices on the bus, a mechanism that determines which device may use the bus is needed. Fundamentally, this is called bus arbitration.

A PCI device also may have a need to communicate with its driver, which is a piece of software that runs in the operating system’s context and operates the device. Since there are a lot of operating systems, that same device may have several different versions of drivers, one for each OS. To communicate with the driver, the device holds an IRQ Line (Interrupt Request) and uses it to signal the CPU that the driver should be called and handle the event.

In this request, which is called an Interrupt, the driver may need to read some data from the device (think about a network packet that arrives to the machine). To pass the data (I/O), a PCI device may have two options, to pass the data by some I/O ports or by sharing a pre-determined memory region. The IRQ line and the addresses of the shared memory region are called the device’s resources.

When an interrupt occurs, the CPU stops the execution of the current software that is running and passes control to the driver. This is called pre-emption. So, the driver handles this event, and a function that is called the ISR (Interrupt Service Routine) is being executed. It reads and writes some data from the device and processes it. When a user’s software will ask for this data, the driver will have it ready.

Ok… let’s get back to PCI itself. PCI is actually the successor of an older bus called ISA, which was slower, and the predecessor of PCI-e (PCI Express ), which is newer and extremely faster. But yet, most of the hardware is designed for PCI and it is still the most common bus in the PC market. Don’t be mislead, even those “On-Board” devices which are so common those days are connected through the PCI bus. PCI slots are used for expansion.

The PCI bus is composed of 32 lines of electricity, each represents a bit, which is the smallest representation of data. It might be 0 or 1. Together, those 32 bits are composing a number in the range of 0 to 4,294,967,296. The PCI bus is working on a clock speed of 33MHz. That means that each line (bit) may be sampled in less than 1/33 of a second.

In the next article, I’ll introduce some new terms like DMA and BIOS.

Thanks for reading!