8th December, 2006

Interrupts and Interrupt-Controllers   

Posted in design, lowlevel, programming | by evilbitz |



Abstract

This article is kind of a continuation for my under-the-hood article series, you can take a look at my previous articles regarding the PCI bus. In this article, I delve further deeper into interrupts, and we’ll explore what exactly happens when an interrupts occurs.

If you are a software developer, this information will help you grasp the PC architecture a little bit better. This article’s intent is to encourage people to explore things and widen their knowledge… I see too many people which are trapped inside their operating system’s environment, especially if they are using the M$ black box. This is the only reason I see for switching to Linux :-) Ok, so let’s start!

Interrupts

An interrupt is basically a signal from the hardware that tells the software to perform an operation. It is handled by the operating system that calls the ISR (Interrupt Service Routine) for that interrupt request (IRQ).

Generally, we distinguish between two different types of interrupts, Edge-Triggered and Level-Triggered. Edge-Triggered interrupts are interrupts which are being caused by changing the bus line level, it is basically a transition from a 1 to 0 or from 0 to 1 (falling-edge and rising-edge repectively). This “old fashion” type of interrupt was used in the ISA bus. The problem with this type of interrupt is that it is difficult to be shared, that means, several devices couldn’t shared the same IRQ line. Level-Triggered interrupts are being caused by raising or lowering the level of the bus line and holding it right there until the interrupt is serviced, they were used in the original PCI bus (and are still being used) as the standard type of interrupt.

Interrupt Controllers

The interrupt controller’s goal is to provide interrupt capabilities to the main processor (CPU) through a single line, when a device issues an interrupt, it is delivered to one of the interrupt controller’s IRQs (pins), from there the interrupt is generated in the CPU, which, in turn, checks with the interrupt controller for the source of the interrupt through a special register which are being hold at and managed by the interrupt controller. Old interrupt controllers, such as the PIC (Programmable Interrupt Controller) provided interrupt-priority, interrupt-masking and general flexibility for dealing with interrupts in the platform. Old devices where programmed to use fixed IRQs and problems arose when two or more devices shared the same IRQ, if the PIC was programmed to be used in the edge-triggered mode, then serious conflicts could cause the system to hang or not function at all. Well… In the edge-triggered mode interrupt actually could be shared if the devices were specially built for this event. But since the operating system must run all the ISRs that exists in the chain that is associated with that specific IRQ that is signaled, it is not so effective after all.

In the Level triggered mode, the PIC knew how to share IRQs between different devices, but sharing interrupts is not a good deal in any case, this of course leads to performance issues and faults that are being caused by poorly written device drivers.

Consider the following scenario: Device A, which shares his IRQ with Device B, signals its driver, the interrupt is issued and the operating system processes it. The chain of ISRs contains two different ISRs. First the driver for Device B is processing the interrupt becuase he is first in the chain, and actually decides that he is going to handle the interrupt (because its poorly written ISR handles any interrupt). The operating system sees that the interrupt was handled and stops executing the ISR-chain and acknowledges the PIC. After some time Device A sees that the interrupt wasn’t handled by its driver and issues the same interrupt again. This scenario leads to interrupt storms or causes Device A to stop function.

To solve this problem and allow more flexibility, Advanced Programmable Interrupt Controllers (APICs) were introduced, they contain more IRQs and their function is better adapted to the operating system. Windows, for example uses its IRQL mechanism to mask interrupts in the APIC, this is accomplished by a single mov assembly instruction, instead of runing several I/O port instructions (such as in or out) in the case of using a PIC.

APICs are being used in Multiprocessor environments, each CPU has its Local APIC and another controller, called I/O APIC actually routes interrupts from the bus to the LAPIC. This allows greater flexibility, and sharing interrupts is no longer an issue, since each LAPIC has 24 IRQs. Another advantage is that each APIC has it’s own timer, this allows each CPU to better schedule the CPU time distributed between threads in quantum units. APICs are also supports IPIs (Inter-Processor-Interrupt), a way of one processor to interrupt another processor. IPIs are being used for synchronization and cache-coherency. The I/O APIC’s function is to distribute interrupts between the CPUs in a multi-processor environment.

Final Words

If you bared with me to this point, It is surely admirable! even I couldn’t bare with myself writing this post :-)

Anyway, we saw how interrupts are being generated, how they are routed at some platforms which are based on the PIC and APIC implementation and we also took a look at how the operating system handles interrups. I hope this post was enjoyable for you.

Evilbitz.



There are currently 15 responses to “Interrupts and Interrupt-Controllers”

Why not let us know what you think by adding your own comment! Your opinion is as valid as anyone elses, so come on... let us know what you think.

  1. 1 On December 10th, 2006, Zefram said:

    Level-triggered interrupts are not as good for sharing as they arereputed to be. Other things being equal, they share less well thanedge-triggered interrupts.

    With either triggering mode it is perfectly possible to share interrupts,provided that the devices and the OS are designed to allow sharing.Many older ISA devices assume that they have exclusive use of the IRQline, which causes problems, but well-behaved ISA devices share just fine.The PCI standard requires devices to allow sharing, which is why thisproblem doesn’t arise with PCI. That aspect of interrupts is completelyindependent of the triggering mode used on the interrupt line.

    Consider what happens if there is a device sharing the interrupt linethat the OS doesn’t know how to service. If this device generates aninterrupt, this will look like a spurious interrupt to the OS. If theinterrupt line is edge-triggered then this doesn’t matter: the interrupthandlers each get invoked once and do nothing, and then everything carrieson as normal. If the interrupt line is level-triggered, however, theunknown device holds the interrupt line asserted indefinitely. All thehandlers get invoked, none of them do anything, and the interrupt lineis still asserted. Now none of the devices can successfully signal aninterrupt: their signals are not noticed, because they can’t change theline’s state.

    What I described as happening with a device that the OS doesn’t understandcan also happen with a device that gets into a state that its driverdoesn’t expect, or a device that locks up due to a fault, or a device thatgenerates a spurious interrupt due to a fault. Unhandleable interruptsare fatal to sharing on a level-triggered interrupt line, but merelya small annoyance on an edge-triggered line. Level triggering is thusinherently brittle. The risk of this failure mode is surely much moreimportant than the small efficiency gain that level triggering provideswhen everything is working as intended.

    Even when everything works correctly, level triggering destroys some ofthe flexibility that the OS would otherwise have in how to drive devices.With edge triggering, the OS has the option to run some of the interrupthandlers immediately and defer others. Devices can be handled withvarying priorities; drivers can compete with ordinary processes forCPU time. This is possible because an as-yet-unhandled interrupt from alow-priority device does not interfere with the reception and handling oflater interrupts from high-priority devices. With level triggering, anunhandled interrupt from any device effectively blocks interrupts from allothers, so it is necessary to handle all interrupts with high priority.

  2. 2 On December 11th, 2006, evilbitz said:

    Hi Zefram,

    Thanks for your clarification on this matter.

    Edge-Triggered Interrupts are a better choice if proper implementation were done in part of the device maker. But this issue put the whole platform in risk just because of a single device. The same is true for level-triggering devices, the issue you described can be solved by a proper device timeout.

    At the other hand, if the OS doesn’t handle an Edge-Triggered interrupt, an interrupt-storm might occur (described in the post), since the device will continually assert the line over and over again.

    Proper handling of each type of interrupt by the OS might solve these problems. If the OS would profile an Edge-Triggered Interrupt and will decide to mask it if a problem is detected, this issue will be solved, and the same is true for a Level-Triggered Interrupt. Of course that profiling will lead into a great performance degradation.

    Your thoughts?

  3. 3 On December 11th, 2006, Zefram said:

    The interrupt storm that you describe can happen with edge triggering if a device not being serviced feels the need to redundantly signal its interrupt again. That would be a very badly-behaved device. Still, even in an interrupt storm, everything might be OK if the OS can keep up. It’ll spend a lot of time servicing these spurious interrupts, but along the way it’ll service real interrupts too, and if the spurious interrupts are spaced out sufficiently then it can get other work done too. This all depends on the exact rate of spurious interrupts.

    With level-triggered interrupts, any spurious interrupt is effectively an interrupt storm with infinite rate. There is no way that the OS could possibly keep up, because as soon as it has run the interrupt handlers the interrupt is already being signalled again. (Strictly it’s *still* being signalled.)

    The way the 8259A PICs in a PC behave for level-triggered interrupts is instructive here. When the OS reckons it’s handled the interrupt, it signals EOI to the PIC. If the IRQ line level is still high when the EOI is issued then the PIC immediately resignals the interrupt to the CPU. The interrupt is resignalled exactly as fast as the CPU handles it. The OS has no chance to do anything other than handle interrupts.

    In either case, if the OS detects the interrupt storm and masks that interrupt line, it loses the ability to receive interrupts from any other device on the same line. This is not exactly solving the issue. It would let the rest of the system work, but with the loss of those devices.

    So in summary, in both triggering styles a seriously broken device can cause an interrupt storm that makes the interrupt line unusable, but it’s much easier with level triggering. With edge triggering the device has to actually keep toggling the interrupt line, and has to do so faster than the OS can keep up with. With level triggering, however, the automatic behaviour of a single spurious interrupt is a fatal interrupt storm.

    On another issue, you didn’t mention message-signalled interrupts, as used in PCI Express. These are the way of the future. They act like edge-triggered interrupts from the point of view of what we’ve been discussing, and they don’t require a dedicated trace. You get sharing for free.

  4. 4 On December 11th, 2006, evilbitz said:

    Well, I think that you summarized everything pretty well. I would just like to note one last thing about the Edge vs. Level triggering mechanisms. From a perforamce point of view, when using edge-triggering, the OS has to run the *whole* ISR chain in order to see if a shared device had also signaled an interrupt while the OS serviced the first one. This is not happening in the level trigerring mode.

    MSIs (Message-Signalled interrupt) solves all these problems and they will surely replace edge-triggered interrupts and level-triggered interrrupts. The industrly is currently still adopting them, and they’ll become “mainstream” pretty soon. but I didn’t mention them because I wanted the article to be lightweight :)

  5. 5 On March 25th, 2007, tigerstyle said:

    Very Good Article ! … very helpful .. thank you.

  6. 6 On May 25th, 2007, Villainova said:

    “in turn, checks with the interrupt controller for the source of the interrupt through a special register which are being hold at and managed by the interrupt controller.”

    can you elaborate on this, please and thanks.

  7. 7 On July 11th, 2007, Jose said:

    I’m not a computer tech lord, and I certainly don’t understand everything. (I run slackware linux, but I certainly don’t RUN it. :) )

    Anyway, Nice article. Very good for satisfying my curiosity. Although, what is a LAPIC?

  8. 8 On July 24th, 2007, evilbitz said:

    Jose, LAPIC is the abbreviation of Local Advanced Programmable Interrupt Controller (Local APIC). It is part of the CPU in modern architecture, and hence it is called “Local”.

    It means that, LAPIC == APIC.

    The LAPIC has several fucntions that I mentioned in the post itself.

  9. 9 On August 1st, 2007, sushil Mayengbam said:

    … good .. point to point and brief explanation… like Robert Love’s ‘Linux kernel development’. I really enjoy it!

  10. 10 On October 1st, 2007, Abhishek said:

    Nice post. Do you have any idea of using APIC timers in 2.6 kernel. I was looking for a patch for APIC timer for the present kernel (2.6). However, all I got were patches for older kernels. Have the patches been merged into the kernel source tree? Any ideas would be appreciated.

  11. 11 On December 18th, 2007, biz said:

    Good ‘down-to-earth’ article. The comments from Zefram were quite interesting, too.

  12. 12 On July 25th, 2008, quest said:

    is it possible for a programmer to generate interrupt to CPU without using Interrupt Controller? is it going to be very complex to do so? what would have happened if there was no interrupt controller at all?
    please let me know. My appologies if my questions are too silly:(

  13. 13 On July 27th, 2008, evilbitz said:

    quest, I’m not sure I understood you, what exactly are you trying to do?

    You probably can do that with IPIs (Inter-processor interrupts), and you can also use the HPET (High precision event timer) in order to get a single or a periodic interrupt.

  14. 14 On October 30th, 2008, suresh said:

    Very nice article.. I got exactly what i was looking for. Zefram made this article more valuable .Through Zefram’s comments we see that edge -triggered has more advantageous(handling the Unknown device interrupt and interrupt -storm handling,). I have a doubt,despite the advantage of edge -triggered why PCI designers have gone for Level Triggered ?

    ~Suresh

  15. 15 On March 3rd, 2009, jackson said:

    pls i will need a material on interrupt program for my project

Leave a Reply

You must be logged in to post a comment.

Top »
"If you can't join them, beat them!"
Search Evilbitz: