On the design of flexible real-time schedulers

for embedded systems

Hugo Marcondes1, Rafael Cancian2,Marcelo Stemmer2, Antonio Augusto Frohlich1

hugom@lisha.ufsc.br, cancian@das.ufsc.br, marcelo@das.ufsc.br, guto@lisha.ufsc.br

1 Laboratory for Hardware and Software Integration2 System and Automation DepartmentFederal University of Santa CatarinaPO Box 476 - Florianopolis - Brazil

88040-900

Abstract

A major subject in real-time embedded systems is themanagement of the tasks that compose the system re-specting its real-time constraints. This is usually ac-complished by the adequate selection of a schedulingpolicy. This work proposes a design and implemen-tation of real-time schedulers for embedded systemswithin the context of Application Oriented SystemDesign (AOSD), enabling the development of sched-ulers where the policy is detached from the schedulingmechanism. The results shows that such implementa-tion could scale from 8 bits microcontrollers, 32 bitsarchitectures, and to specific hardware implementeddesign.

1 Introduction

Embedded Operating Systems (EOS) should beadapted to provide only the necessary support for awell-defined target-application. Factoring out the op-erating system into selectable and configurable com-ponents is a good way to model and design an EOS.However, in the realm of dedicated systems, an inte-grated design of software and hardware is often in-duced. In order to accomplish with the system re-quirements, the design of such systems need to dealwith a diversity of hardware architectures, as 8-bitsmicrocontrollers to dedicated chips (ASICS), and dig-ital signal processors (DSP). In this scenario, the taskof modeling and implementing components that canbe effectively reused in this distinct architectures is

a challenge.The adaptations needed to an EOS meet the re-

quirements of the target-applications usually spreadsamong the components of the system. As an exam-ple, depending on the application, the EOS may needto support mono task, cooperative tasks, or concur-rently tasks. Only in the later case a task sched-uler is needed, and it may be implemented concern-ing with several features as policies, dynamic prior-ity, preemption, etc. Schedulers also depend on timemanagers like alarms, and alarms depend on timercounters. They also need to know specific informa-tion that belong to the tasks that they will schedule(such as priority, arrival time, deadline, and period).The concurrent tasks also depend on synchronizationmechanisms, such as semaphores and mutexes, whichin turn need the schedulers; if they are real-time syn-chronization mechanisms, then they will depend ontimers as well (i.e. to timeout P operations in asemaphore). This illustrates how related all those ab-stractions are. In this way, successfully adapt them toapplication requirements demand a good engineeringstrategy.

Even in a single family, the adaptation of com-ponents to different execution scenarios may not beeasy. Task schedulers, for instance, have a myriad ofalgorithms and features, including relatively complexand highly specialized real-time policies. It is still abig challenge allow an embedded operating system tosupport any scheduling algorithm (real-time or not),independently of its features, and without requiringadaptations on the rest of the system.

The integrated design of software and hardware on

2009 International Conference on Computational Science and Engineering

978-0-7695-3823-5/09 $26.00 © 2009 IEEE

DOI 10.1109/CSE.2009.407

382

2009 International Conference on Computational Science and Engineering

978-0-7695-3823-5/09 $26.00 © 2009 IEEE

DOI 10.1109/CSE.2009.407

382

embedded systems allows features typically found onoperating systems to be implemented in hardware,by using programmable logic devices or even design-ing ASICs. Nowadays is not rare implementing taskschedulers on hardware, because this is a very oftenused component and it is source of considerable over-head. Indeed, an application oriented operating sys-tem has to allow their components to be implementedon both the domains (software and hardware) in anefficiently way.

All these problems can not be solved only withcareful implementation. They need an appropriatedand ingenious system design. In this paper, we focuson the description of the analysis and modeling of thecomponents related to task schedulers and advancedimplementation aspects that, only together, allow theappropriated solution for these challenges. We useddomain engineering and a method that aggregatesa set of programming paradigms and guide the de-sign of systems adapted to the application. Our con-tributions includes an efficient model and the corre-spondent implementation to adapt schedulers on ap-plication oriented embedded operating systems, alsoallowing its execution on different architectures, in-cluding small 8 bits microcontrollers.

This paper is organized in this way: section 2presents concepts and the description of some sched-ulers, as well as the description of the main techniquesused in this work. The sections 3 and 4 present thedevelopment held and the results obtained, includingthe theoretical model and aspects of the proposedimplementation. Finally, the section 5 presents someconclusions and final considerations.

2 Related Work

The tasks scheduling is considered the heart of a sys-tem and dozens of distinct algorithms have been pro-posed, and most of them are real-time schedulers forspecific applications classes. Many schedulers can bereduced into a simple sort of a ready tasks queue,according to a specific criterion. Indeed, this in-cludes the most known schedulers, such as FIFO,round robin, priority, SPF (Shortest Process First),RM (Rate Monotonic), and EDF (Earliest DeadlineFirst) [7]. However, several other algorithms aremuch more complex than that. Algorithms such asDSS (dynamic sporadic server) and dynamic priorityexchange server [3] need separated queues for peri-odic and aperiodic tasks, and at least one special peri-odic task to deal with the aperiodic tasks with specifictiming rules, consumption, and budgets grant. Algo-

rithms such as Elastic Task Model [6] allow changeson tasks parameters, such as its period, in order toadapt themselves to the current system load. There-fore, to support them in a transparent and efficientway for embedded systems is far from be a simpletask.

Several embedded operating systems already allowthe adaptation of their schedulers, even dynamically.However, this adaptation is always restricted to afew and simple specific algorithms, such as FIFO,round robin, priority, EDF, and RM. Besides that,many real-time operating systems use scheduler algo-rithms that do not considerate the tasks’ deadline, i.e.they use non real-time schedulers to schedule real-time tasks. Finally, in addition to require guaranteeof the tasks’ deadline, many embedded real-time ap-plications require further more from the schedulers.As an example, multimedia applications, require thatthe constant execution rate of audio and video tasks(minimizing the jitter). To deal with this kind of re-quirement is necessary the use of specific algorithms,such as CBS (Constant Bandwidth Server) [6], justbecause usual algorithms do not take the jitter intoaccount. From this we can conclude that there is noadequate support for this kind of application whenthe EOS does not provide specific schedulers, andthis is the case of many EOS, including the real-timeones.

Several researches on hardware/software co-designfor real-time systems have been proposed in this andin last decades. Hardware support for task schedulerswas proposed, among others, by [9], that had im-plemented a cyclical schedule, and by [10], that hadimplemented the RM and EDF priority algorithms.Beyond the support for tasks scheduling, [8] had de-veloped hardware support for time and events man-agement, because these activities are very often onthe real-time systems and have a high intrinsic par-allelism. However, this support is limited to fixedpriority schedules. The HThread project [2] proposesa programming model that allows tasks implementedon hardware interact with tasks at software, by theimplementation of schedulers and synchronization de-vices on both the domains (hardware and software).Others kinds of support had also been proposed, likememory management [13] and resource access pro-tocols [1], that had implemented the priority inher-itance protocol to prevents deadlocks and unlimitedtask blocking. Supports that are more complete in-clude, beyond the scheduling, inter process commu-nication, interrupts management, resources manage-ment, synchronization and time management. This

383383

support on hardware is usually called real-time Unit(RTU).

In this scenario, the EPOS (Embedded Parallel Op-erating System) raises up as one viable choice of mul-tiplatform real-time operating system for embeddedsystems. The EPOS includes frameworks and toolsfor operating system generation, and it is result ofthe Application-Oriented System Design (AOSD) [4],that combines several design paradigms that aimsguide the development of high adaptable e reusablecomponents. The AOSD brings innovations as sce-nario adapters [5] e hardware mediators [11] that al-low high efficiency on automatic generation of appli-cation dedicated systems.

EPOS does not address only the automatically gen-eration of the software support for an specific ap-plication, but also the support for hardware blocks(IPs - Intellectual Properties) necessary and sufficientto them, i.e., the application oriented and automaticgeneration of SoCs (Systems-on-a-chip). The exten-sions on EPOS to SoCs generation are based on theabstraction concepts, hardware mediators and IPs,associating one IP for each mediator [12]. Currentlythe EPOS has functional support for several architec-tures such as IA32, PPC, SparcV8, MIPS, and AVR.

3 Analysis and Design

The analysis and design process begin with domainengineering, following the guidelines of the AOSDmethodology, which enables the identification of themain commonalities and differences between the con-cepts that compose the domain. Using this strategythe main entities related to real-time scheduling wereidentified. The figure 1 presents the design of suchentities.

In this design, the task is represented by the classThread and defines the execution flow of the task,implementing the traditional functionality as the sus-pend and resume operations. This class models onlyaperiodic tasks. Periodic tasks, a common abstrac-tion of real-time systems, are in fact a specializationof the Thread class which aggregate the mechanismsrelated to the re-execution of the task periodically,using the Alarm abstraction, responsible for reacti-vating the task when a new period begins. The Alarmabstraction uses the Timer hardware responsible tomanage the timing duties of the system.

The classes Scheduler and SchedulingCriteriadefine the structure that realizes the task scheduling.Traditional design and implementations of schedulingalgorithms are usually done by a hierarchy of special-

- times: int- period: int

Alarm

+ Alarm()

Thread

+ yield()+ suspend()+ resume()+ join()

PeriodicThread

+ wait_next()

Scheduler

+ insert(thr: Thread)+ remove(thr: Thread): Thread+ suspend(thr: Thread)+ remove(thr: Thread)+ chosen(): Thread+ choose(): Thread+ choose(Thread): Thread+ choose_another(): Thread

SchedulingCriteria

+ operator int()

Priority

FCFS RoundRobin SJF

EDF

- deadline: intRealTime

RM

<<hardware>>Timer

+ interrupt()

11..*

EnergyAwareElastic

Elastic

Preemption

AdmissionControlRelativeQueue

CPU_Context

Stack

1

1

Figure 1: Proposed task scheduling design

ized classes of an abstract Scheduler class, which canbe further specialized to bring new scheduling poli-cies to the system. In order to reduce the complex-ity of maintenance of the code (generally present insuch hierarchy of specialized classes), as well as topromote its reuse, our design detaches the schedul-ing policy (criteria) from it’s mechanisms (lists im-plementations) and also detaches the scheduling cri-teria from the thread it represents. Such division ofresponsibilities is yield from the domain engineeringprocess.

Such separation of the mechanism from thescheduling policy was fundamental for the construc-tion of the scheduler in hardware. In fact, thehardware Scheduler component implements onlythe mechanisms that realize the ordering of thetasks, based on the selected policy. In this sense,the same hardware component can realize distinctpolicies, without any hardware reconfiguration, asthe definition of the policy is confined in theSchedulingCriteria component. This is achievedby the isolation of the element’s comparison algo-rithm of the scheduler in the criteria, analogous tothe separation of algorithm and the elements of datastructures defined in the Stl library.

Additionally to the analysis and domain engineer-ing process, several characteristics were identified asconfigurable features of those components. In fact,such characteristics represent fine variations withinan entity of the domain, which can be set in or-der to change slightly the behavior of the compo-nent. Among such configurable features, preemp-tion, admission control of tasks, energy consumption

384384

and changes in tasks’ information were identified asslightly variations of the scheduler. Admission con-trol of tasks (i.e. based on CPU utilization of thecurrent set of tasks), as well as the consideration ofenergy consumption of energy could be evaluated asconfigurable features responsible by the implementa-tion of quality of service policies (QoS). The capa-bility that a scheduler have to change the propertiesof tasks that are been used is useful for several morecomplex algorithms. As seen on section 2, elasticscheduling algorithms (as the elastic task model), as-sume that the period of a task could be changed, asthe CPU utilization rate are getting higher or lower.Others schedulers, as the CBS and DSS (section 2)have analogous behavior. Such characteristic is de-signed as a configurable features that are applied tothe SchedulingCriteria related to periodic tasks,as well as the PeriodicThread, enabling the func-tions to change the period of one task, once the sched-uler requests. In this sense, algorithms that are morecomplex could be supported and adapted without in-curring new specialization of classes.

: Timer : CPU : Alarm : Scheduler running: Thread new_thr: Thread

interrupthandler()

: Thread

[QUANTUM_EXCEEDED]reschedule()

running = chosen()

new_thr = choose()

[running != new_thr]switch_threads(running, new_thr)

state(READY)

state(RUNNING)

switch_context(running, new_thr)

Figure 2: Task rescheduling sequence diagram

In order to illustrate the interactions between thecomponents of the proposed design, the figure 2presents the interactions of the components duringthe rescheduling, occurred when the time slice of thecurrent task expires. In this context, the Timer is re-sponsible for generating periodic interruptions, whichare counted by the Alarm. When the CPU timeslice (quantum) given to the current running threadis expired, the Alarm invokes the Thread methodresponsible for rescheduling the tasks. Then, therescheduling method verifies which is the actual run-ning thread, as well as to verify which one should bethe next to run, invoking the choose() method fromthe Scheduler. This method returns a pointer to thethread that should be running. The actual runningthread pointer is compared with the pointer returnedby the choose() method, in order to verify if a switch

context of the CPU is necessary. If a switch contextis necessary, the threads states are updated and thecorresponding method for switch the CPU context isinvoked.

4 Implementation and Results

This section presents the implementation details ofthe main components of the proposed schedulerschema, specially the implementation of the schedul-ing mechanism in the software and hardware domain.The main scheduling policies implemented throughthe SchedulingCriteria are also presented.

4.1 Software Scheduler

The implementation of the software scheduler followsthe traditional design of lists. Such list implementa-tion its realized as a conventional ordering list of itselements, as well as a relative list, where each ele-ment stores its ordering parameter relative with itspredecessor. In this sense each element will hold thedifference of its ordering parameter from the previ-ous element, and so on. Such kind of implementationis necessary when the scheduling policy has dynamicpriority that increases over time, as the very knownEDF policy, as an example. In such policy, as theabsolute deadline is always a crescent value, the useof a conventional ordering, using the absolute dead-line will lead to an overflow of the variable (whichcan occur in a few hours on 8 bits microcontrollers).Instead of, the use of a relative queue insures thatthe deadline is always stored relatively to the currenttime, and in this way, the variable will never overflow.

Independently of the use of relative queues orconventional one, the criterion used by the or-dering algorithm of the queue is realized by theSchedulingCriteria. In general, this componentcan be visualized as a specialization of the inte-ger type, which defines the ordering of the queue.Policies that are more complex can be establishedby overloading its arithmetical operators. As ex-ample, in the case of multi-queues algorithms, aSchedulingCriteria can encapsulate two parame-ters for ordering: the identification of the queue, andthe priority of the element inside that queue, as wellas overload the comparison operator less-equal (≤)in order to evaluate both parameters when the ele-ments are compared to establish its position insidethe queue implemented on the Scheduler compo-nent. This approach allows the efficient implemen-tation of more complex scheduling algorithms.

385385

4.2 Hardware Scheduler

The component Scheduler was also implemented inthe hardware domain. The figure 3 illustrates theorganization of the logical blocks of this component.

INTE

RFAC

E

Controller

Memory

INSERT

REMOVE

SUSPEND

RESUME

CHOOSE

CHOSEN

CHOOSE_ANOTHER

Resource Control

RANK

V NEXT

POINTER

PREV

TAILHEAD

clockcommand

input 0status

input 1

outputinterrupt

SCHEDULER_IP

PLAT

FORM

GLU

E LO

GIC

Figure 3: Block diagram of the proposed componentin hardware.

The implementation of the scheduler in hardwarefollows a well-defined structure. It has an internalmemory that implements an ordered list. One mod-ule (Controller) is responsible for interpreting allthe data received by the interface of the component inhardware and then to activate the process responsiblefor implementing the functionality requested by theuser (through the command interface register). Thisimplementation, as the software counterpart, realizesthe insertion of its elements already in order, that is,the queue is always maintained ordered, following theinformation that the SchedulingCriteria provides.

It worth’s highlight two aspects of the implemen-tation of this component regarding its implementa-tion on hardware, especially for programmable logicdevices. Both of these aspects are related to the con-straints in terms of resources of such devices. Ideally,a hardware scheduler should exploit as most the in-herent parallelism of the hardware resources. How-ever, such resources are very expensive, especiallywhen the internal resources are used to implementseveral parallel bit comparators to search elementson the queue, as well as to find the insertion posi-tion of an element in queue. Moreover, the use of 32bits pointers to reference the elements stored on thelist (in this case Threads) becomes extremely costlyfor implementing these comparators. On the otherside, the maximum number of tasks in an embeddedsystem is usually known at design time, and for thatreason, the resources usage of this component couldbe optimized by implementing a mapping betweenthe system pointer (architecture word-size) and aninternal representation that uses only the necessary

number of bits, taking into account the maximumnumber of tasks running on the system.

Another aspect is related to the search of the el-ement’s insertion position. Ideally, such searchingcould be implemented through a parallel comparisonbetween all elements on the queue, in order to findthe insertion point in only one clock cycle. However,such approach, besides increasing the consumptionof the resources, as the number of tasks increase itcould lead to a very high critical path delay on thesynthesized circuit, and thus, reduce the operatingfrequency of the component.

By this reason, the insertion of elements was im-plemented doing a sequential search of the insertionposition of the element, which will take N cycles inthe worst-case. Besides the fact that this approachinserts variable time for this operation, such varia-tion is hidden by the fact that the insertion couldbe realized in parallel to the software running on theCPU.

4.3 Evaluation

The evaluation of the proposed scheduler was real-ized implementing a synthetic real-time application,where a set of periodic tasks was defined. The im-plemented components were configured using the ap-propriate tools, which generate a set of configurationparameters binding the interface of the componentwith its implementation (that could be realized assoftware or as hardware, through the binding of theinterface with its mediator).

The application was compiled for the PowerPC (32bits) and AVR (8 bits) architecture, using the Edf,Rate Monotonic e Priority. The table 1 presentthe footprint of the application for each selected pol-icy and architecture. The tests also validated theimplementation of each scheduling policy.

Ppc32 Avr8.text .data .text .data

EDF 51052 300 49246 853Rate Monotonic 47908 272 36800 1003Priority 47864 272 36790 1003

Table 1: Test application footprint

The tests were realized also using the scheduler inhardware. In this case, the experimentation platformwas a Virtex4 FPGA, which combines on PowerPC405 processor and logic cells, enabling the rapid pro-totyping of dedicated hardware accelerators.

386386

# Max. Tasks Logic Usage Slices Max. Freq.2 5% 326 214.6 Mhz4 10% 551 161.5 Mhz8 19% 1078 138.8 Mhz16 36% 2015 123.4 Mhz24 51% 2833 114.6 Mhz32 73% 3997 113.4 Mhz48 103% 5665 82.0 Mhz

Table 2: FPGA resource utilization of the Schedulercomponent

The FPGA used on the experimentation platform(ML403) was the XC4VFX12 that provides 5,412 slicesof logic blocks for the implementation of the acceler-ators. The table 2 shows the consumed area in thisFPGA, accordingly with the configured number ofmaximum task instantiation.

5 Conclusions

This paper presented a design of a flexible real-timescheduler. The use of refined techniques as domainengineering enabled the isolation of the differences ofseveral scheduling policies, enabling a better reuseof the design artifacts (as scheduling policies andscheduling mechanisms), as well as providing a plat-form not only to design real systems, but also todo experimentation of new real-time scheduling al-gorithms. The tests show the feasibility of deploythe scheduler design on architectures that range fromsmall 8 bits microcontrollers to 32 bits architecturesand even dedicated hardware.

References

[1] B. Akgul. Hardware support for priority in-heritance. In Kluwer Academic Publishers, edi-tor, 24th IEEE International Real-Time SystemsSymposium, 2003.

[2] E. Anderson, J. Agron, W. Peck, J. Stevens,F. Baijot, E. Komp, R. Sass, and D. Andrews.Enabling a uniform programming model acrossthe software/hardware boundary. In Field-Programmable Custom Computing Machines,2006. FCCM ’06. 14th Annual IEEE Symposiumon, pages 89–98, 2006.

[3] Giorge Buttazzo. Hard Real-Time ComputingSystems. Kluwer Academic Publishers, 1997.

[4] A. A. Frohlich. Application-Oriented OperatingSystems. PhD thesis, Sankt Augustin: GMD -Forschungszentrum Informationstechnik, 2001.

[5] A. A. Frohlich and W. Schroeder-Preikschat.Scenario adapters: Efficiently adapting compo-nents. In Proceedings of 4th World Multiconfer-ence on Systemics, Cybernetics and Informatics,Orlando, USA, 2000.

[6] G. Lipari G.C. Buttazzo and L. Abeni. Elastictask model for adaptive rate control. In 19thIEEE Real-Time Systems Symposium, pages286–295, Madrid, Spain, 1998.

[7] D. R. CHOFFNES Harvey DEITEL, Paul DEI-TEL. Sistemas operacionais. Prentice Hall,2005.

[8] P. Kohout and B. Jacob. Hardware support forreal-time operating systems. In Proceedings ofCODES - ISSS’03, Newport Beach, CA - USA,2003.

[9] V. Mooney and G. De Micheli. Hardware/soft-ware codesign of run-time schedulers for real-time systems. In Proceedings of Design Automa-tion of Embedded Systems, pages 89–144, 2000.

[10] M. Shalan P. Kuacharoen and V. Mooney. Aconfigurable hardware scheduler for real-timesystems. In Proceedings of International Confer-ence on Engineering of Reconfigurable Systemsand Algorithms - ERSA’03, 2003.

[11] F. V. Polpeta and A. A. Frohlich. Hardwaremediators: a portability artifact for component-based systems. In Proceedings of InternationalConference on Embedded and Ubiquitous Com-puting, volume 3207, pages 271–280, 2004.

[12] F. V. Polpeta and A. A. Frohlich. On the au-tomatic generation of soc-based embedded sys-tems. In Proceedings of the 10th IEEE Interna-tional Conference on Emerging Technologies andFactory Automation, 2005.

[13] M. Shalan and V. Mooney. A dynamic mem-ory management unit for embedded real-timesystem-on-a-chip. In Proceedings of Interna-tional Conference on Compilers, Architectureand Synthesis for Embedded Systems, pages 180–186, 2000.

387387