Memory Management Function
This article explains the memory management function of T-Kernel.
T-Kernel is a real-time OS for embedded devices. Limited resources must be utilized effectively in embedded systems. An easy example is memory.
Information-related OSs have memory in gigabytes (GB) or are designed to use virtual memory, so there is almost no concern about memory exhaustion. However, this is not the case in embedded devices. Even if you try to allocate memory during execution, you may not be able to acquire it due to insufficient memory, so the program needs to be developed on such an assumption.
Therefore, T-Kernel provides the following two functions for managing memory.
- Variable Length Memory Pool
- Fixed-length memory pool
In this article, I’ll explain how to use them.
Variable Length Memory Pool
Variable-length memory pools provide a function to manage memory blocks of arbitrary size, similar to the malloc function in the C language library. The main differences from malloc are as follows
- Multiple memory pools can be provided for different uses.
- You can specify a timeout if a memory block cannot be acquired.
Since multiple memory pools are available in T-Kernel, for example, separate memory pools can be allocated for processes that can wait for a shortage of memory and for critical processes that are not allowed to be interrupted due to a shortage of memory, thus making it possible to build a system in which critical processes are less likely to be affected by other processes.
【Figure 1: Variable-length memory pool and malloc (1)】
If memory cannot be acquired, some information-based operating systems may reserve memory space by saving the memory contents to secondary memory (page out) or automatically perform garbage collection if memory is insufficient. Although these are convenient functions, they are not suitable for embedded devices that require real-time performance because there is a risk of waiting for a long time without knowing when the memory will be acquired.
On the other hand, T-Kernel has a timeout function in the system calls for memory acquisition to enable clear control of the behavior in case memory cannot be acquired, and it is not suitable for embedded devices that need real-time performance. Since the timeout time can be specified, the following actions can be selected as the actions when memory cannot be acquired.
- If the memory cannot be acquired, the error is terminated immediately.
- Wait until memory is acquired.
- If it cannot acquire memory after waiting for the specified time, it exits with an error.
In this way, the maximum latency can be controlled by the application in the event of failure to acquire memory, thus ensuring real time for the system as a whole.
【Figure 2: Variable-length memory pool and malloc (2)】
Listing 1 shows a real-world example of using a variable-length memory pool.
【Listing 1: Example of Variable Length Memory Pool Usage】
#include <basic.h>
#include <tk/tkernel.h>
#include <tm/tmonitor.h>
#define HIST_NO (10) /* Number of histories to save */
#define PROMPT "> " /* prompt */
/* Command List */
#define CMD_EXIT "exit" /* end */
#define CMD_HISTORY "history" /* history indication */
#define CMD_REQUEST "request" /* request to start a conversion process */
#define CMD_DISPLAY "display" /* Conversion result display */
#define CMD_BANG '#' /* command replacement control */
#define TMO_GETBLK (3000) /* Waiting time in case of shortage */
/* variable length memory pool */
#define BUFSZ (80) /* Input buffer size */
#define MPLSZ ((BUFSZ*HIST_NO)/2) /* Overall MPL size */
/* ↑ I'm deliberately reducing the size by a factor of two in order to reduce the size. */
LOCAL ID mplid; /* variable length memory pool ID */
LOCAL B* hist[HIST_NO]; /* Pointer to history */
LOCAL INT idx; /* Next History Location */
#define IDX(x) ((x)%HIST_NO) /* For specifying the history array */
IMPORT char* strcpy( char *dst, const char *src );
IMPORT size_t strlen( const char *s );
IMPORT int strcmp( const char *s1, const char *s2 );
#define isdigit(c) (('0'<=(c))&&((c)<='9'))
INT atoi( B *str ) /* String → Integer */
{
INT i = 0;
while( isdigit(*str) ){
i = i*10 + (*str++ - '0');
}
return i;
}
void itoa3( INT val, B *str ) /* Integer → String(3 digits) */
{
INT i;
for( i = 2; i >= 0; i-- ){ /* Maximum 3 trusses */
*(str+i) = val % 10 + '0';
val /= 10;
}
/* Note that '\0' doesn't add it! */
}
ER getCommand(B **cmdBuf ) /* Entering commands */
{
UB buf[BUFSZ]; /* Fixed buffer for input */
B *blk;
INT size; /* Number of characters entered */
ER ercd;
tm_putstring( (UB*)PROMPT ); /* Output a prompt */
size = tm_getline(buf); /* Input from the console */
if( size == 0 ){ /* Line feed only */
return 0;
}
if( size < 0 ){ /* Ctrl-C Enter → Exit */
strcpy( buf, CMD_EXIT );
size = sizeof(CMD_EXIT);
}
size++; /* Add a terminating '\0' minute */
ercd = tk_get_mpl( mplid, size, &blk, TMO_GETBLK );
if( ercd == E_OK ){
strcpy( blk, buf ); /* Save new input */
*cmdBuf = blk; /* return a memory block */
}else{
tm_putstring( "Can't get memory block!\n" );
size = ercd;
}
return size; /* Input command size */
}
ER replaceCommand( B **p_cmdBuf ) /* Command Replacement */
{
INT reqIdx; /* Requested history */
B *cmdBuf = *p_cmdBuf; /* The command you entered */
B *newCmd; /* The area of the replacement */
INT size; /* Size of the replacement */
ER ercd;
if( *cmdBuf != CMD_BANG ){ /* replacement not required */
return E_OK;
}
cmdBuf++;
if( *cmdBuf == CMD_BANG ){ /* one year ago */
reqIdx = idx - 1;
}else if( isdigit(*cmdBuf) ){ /* + "figure" */
reqIdx = atoi(cmdBuf) - 1 + HIST_NO;
}else if( (*cmdBuf == '-') && isdigit(*(cmdBuf+1)) ){
reqIdx = idx - atoi(cmdBuf+1);
}
if( (reqIdx < (idx - HIST_NO))||(idx <= reqIdx) ){
tm_putstring( "Out of range!\n" );
return E_PAR;
}
size = strlen(hist[IDX(reqIdx)]) + 1; /* Size to be secured */
ercd = tk_get_mpl( mplid, size, &newCmd, TMO_GETBLK );
if( ercd < 0 ){
tm_putstring( "Can't get memory block!\n" );
}else{
tk_rel_mpl( mplid, *p_cmdBuf ); /* Discard the replacement string. */
*p_cmdBuf = newCmd;
strcpy( newCmd, hist[IDX(reqIdx)] );
tm_putstring( newCmd );
tm_putstring( "\n" );
}
return ercd;
}
void commandHistory(void) /* View history */
{
B preStr[] = "[000] ";
INT i;
for( i = idx - (HIST_NO-1); i <= idx; i++ ){
if( *hist[IDX(i)] != '\0' ){
itoa3( i - (HIST_NO-1), &preStr[1] );
tm_putstring( preStr );
tm_putstring( hist[IDX(i)] );
tm_putstring( "\n" );
}
}
}
ER commandRequest( B* cmd ) /* conversion request */
{
/* Added in Listing 2 */
return E_OK;
}
void commandDisplay(void) /* Display the conversion result */
{
/* リスト2で追加 */
}
ER executeCommand( B *cmdBuf ) /* Command Execution */
{
ER ercd = E_OK;
if( !strcmp( cmdBuf, CMD_HISTORY ) ){ /* View history */
commandHistory();
}
else if( !strncmp( cmdBuf, CMD_REQUEST, sizeof(CMD_REQUEST)-1 ) ){
ercd = commandRequest( cmdBuf ); /* conversion request */
}
else if( !strcmp( cmdBuf, CMD_DISPLAY ) ){
commandDisplay(); /* Display the conversion result */
}else{
ercd = E_PAR;
tm_putstring( "NO SUCH COMMAND\n" );
}
return ercd;
}
void createTranslateTask(void)
{
/* Added in Listing 2 */
}
EXPORT INT usermain( void )
{
CONST T_CMPL cmpl = { NULL, TA_TFIFO, MPLSZ };
B *cmdBuf;
INT i;
ER ercd;
/*--------------*/
/*initialization*/
/*--------------*/
mplid = tk_cre_mpl( &cmpl );
for( i = 0; i < HIST_NO; i++ ){
tk_get_mpl( mplid, 1, &cmdBuf, TMO_FEVR );
*cmdBuf = '\0';
hist[i] = cmdBuf; /* Add a dummy memory block. */
}
idx = HIST_NO;
createTranslateTask(); /* Creating tasks for conversion */
/*--------------*/
/* main loop */
/*--------------*/
while( 1 ){
ercd = getCommand( &cmdBuf ); /* Input from the console */
if( ercd <= 0 ){ /* Errors or line breaks only */
continue;
}
if( !strcmp( cmdBuf, CMD_EXIT ) ){ /* Final confirmation. */
break;/*while*/ /* → Escape from the loop. */
}
ercd = replaceCommand( &cmdBuf ); /* Replacement for rerun */
if( ercd < 0 ){
tm_putstring( "Replace error\n" );
continue;
}
tk_rel_mpl( mplid, hist[IDX(idx)] );
hist[IDX(idx)] = cmdBuf; /* Replacing commands */
executeCommand( cmdBuf ); /* Command Execution */
idx++; /* Next History Location */
}/*while*/
/*--------------*/
/* 終了処理 */
/*--------------*/
for( i = 0; i < HIST_NO; i++ ){
tk_rel_mpl( mplid, hist[i] ); /* Free the memory blocks. */
}
tk_del_mpl( mplid ); /* Removing the memory pool */
return 1;
}
List 1 is a program that provides a simple history function for strings entered from the console. You can save a string(*1) entered from the console and display the list later, or replace it with another string previously entered.
When a string is entered, the system acquires a memory block of the required size from the variable-length memory pool and stores it there. Also, when a new string is entered, the memory block containing the old input string is released so that there is no shortage of memory.
There are two commands for the history function.
| exit | Exit the program. |
|---|---|
| history | Displays the saved history. |
Also, the following will replace the previously entered string (command).
| ## | Replace the previous command. |
|---|---|
| #[figure] | Replace the command with the history number shown in [figure]. |
| #-[figure] | Replace the previous command with the one indicated by [figure]. |
(*1) To simplify implementation, we use T-Monitor’s tm_getline() to input data from the console. The buffer for tm_getline() is only 80 bytes, so do not input more than 80 bytes and be sure to hit [Enter] before that. If you type too many characters, you can delete them by pressing [DEL] or Ctrl+H.
Fixed Length Memory Pool
Fixed-length memory pools are for managing memory blocks of a fixed size. It is basically the same function as the variable length memory pool, except that the memory blocks are fixed in size.
However, fixed-length memory pools have the advantage that memory blocks can be managed more efficiently because there is no memory fragmentation like in the case of variable-length memory pools. In addition, fixed-length memory pools do not need to manage the size of memory blocks, so the size of the management area is smaller and processing is slightly faster.
Furthermore, there are many applications in real programs that make heavy use of fixed-size memory blocks. For example, when processing incoming packets from the network or disk sectors, a fixed length may be sufficient. Also, when communicating with small input and output devices, such as sensors, data is often set to a fixed size and a fixed format packet to send and receive.
Naturally, the size of the memory pool varies with the application. By providing separate fixed-length memory pools of different sizes for each application, it is possible to implement the system in such a way that a lack of memory for one process does not cause a lack of memory for a completely unrelated process, thereby increasing the stability of the system.
Listing 2 shows an example of using a fixed-length memory pool.
【Listing 2: Example of Fixed Length Memory Pool Usage】
#include <basic.h>
#include <tk/tkernel.h>
#include <tm/tmonitor.h>
#define HIST_NO (10) /* Number of histories to save */
#define PROMPT "> " /* prompt */
/* Command List */
#define CMD_EXIT "exit" /* end */
#define CMD_HISTORY "history" /* history indication */
#define CMD_REQUEST "request" /* request to start a conversion process */
#define CMD_DISPLAY "display" /* Conversion result display */
#define CMD_BANG '#' /* command replacement control */
#define TMO_GETBLK (3000) /* Waiting time in case of shortage */
/* variable length memory pool */
#define BUFSZ (80) /* Input buffer size */
#define MPLSZ ((BUFSZ*HIST_NO)/2) /* Overall MPL size */
/* ↑ I'm deliberately reducing the size by a factor of two in order to reduce the size. */
/* fixed length memory pool */
#define MPFSZ sizeof(T_MSGPKT) /* block size */
#define MPFNO (4) /* Number of blocks */
LOCAL ID mplid; /* variable length memory pool ID */
LOCAL ID mpfid; /* fixed length memory pool ID */
LOCAL ID mbfid; /* Message Buffer ID */
LOCAL ID mbxid; /* mailbox ID */
typedef struct { /* between request-display */
T_MSG msgque; /* converted and forwarded packet */
B bstr[32+1]; /* ← The body of the conversion result */
} T_MSGPKT;
LOCAL B* hist[HIST_NO]; /* Pointer to history */
LOCAL INT idx; /* Next History Location */
#define IDX(x) ((x)%HIST_NO) /* For specifying the history array */
IMPORT char* strcpy( char *dst, const char *src );
IMPORT size_t strlen( const char *s );
IMPORT int strcmp( const char *s1, const char *s2 );
#define isdigit(c) (('0'<=(c))&&((c)<='9'))
INT atoi( B *str ) /* String → Integer */
{
INT i = 0;
while( isdigit(*str) ){
i = i*10 + (*str++ - '0');
}
return i;
}
void itoa3( INT val, B *str ) /* Integer → String(3 digits) */
{
INT i;
for( i = 2; i >= 0; i-- ){ /* Up to 3 digits */
*(str+i) = val % 10 + '0';
val /= 10;
}
/* Note that '\0' doesn't add it! */
}
ER getCommand(B **cmdBuf ) /* Entering commands */
{
UB buf[BUFSZ]; /* Fixed buffer for input */
B *blk;
INT size; /* Number of characters entered */
ER ercd;
tm_putstring( (UB*)PROMPT ); /* Output a prompt */
size = tm_getline(buf); /* Input from the console */
if( size == 0 ){ /* Line feed only */
return 0;
}
if( size < 0 ){ /* Ctrl-C Enter → Exit */
strcpy( buf, CMD_EXIT );
size = sizeof(CMD_EXIT);
}
size++; /* Add a terminating '\0' minute */
ercd = tk_get_mpl( mplid, size, &blk, TMO_GETBLK );
if( ercd == E_OK ){
strcpy( blk, buf ); /* Save new input */
*cmdBuf = blk; /* return a memory block */
}else{
tm_putstring( "Can't get memory block!\n" );
size = ercd;
}
return size; /* Input command size */
}
ER replaceCommand( B **p_cmdBuf ) /* Command Replacement */
{
INT reqIdx; /* Requested history */
B *cmdBuf = *p_cmdBuf; /* The command you entered */
B *newCmd; /* The area of the replacement */
INT size; /* Size of the replacement */
ER ercd;
if( *cmdBuf != CMD_BANG ){ /* replacement not required */
return E_OK;
}
cmdBuf++;
if( *cmdBuf == CMD_BANG ){ /* one year ago */
reqIdx = idx - 1;
}else if( isdigit(*cmdBuf) ){ /* + "figure" */
reqIdx = atoi(cmdBuf) - 1 + HIST_NO;
}else if( (*cmdBuf == '-') && isdigit(*(cmdBuf+1)) ){
reqIdx = idx - atoi(cmdBuf+1);
}
if( (reqIdx < (idx - HIST_NO))||(idx <= reqIdx) ){
tm_putstring( "Out of range!\n" );
return E_PAR;
}
size = strlen(hist[IDX(reqIdx)]) + 1; /* Size to be secured */
ercd = tk_get_mpl( mplid, size, &newCmd, TMO_GETBLK );
if( ercd < 0 ){
tm_putstring( "Can't get memory block!\n" );
}else{
tk_rel_mpl( mplid, *p_cmdBuf ); /* Discard the replacement string. */
*p_cmdBuf = newCmd;
strcpy( newCmd, hist[IDX(reqIdx)] );
tm_putstring( newCmd );
tm_putstring( "\n" );
}
return ercd;
}
void commandHistory(void) /* View history */
{
B preStr[] = "[000] ";
INT i;
for( i = idx - (HIST_NO-1); i <= idx; i++ ){
if( *hist[IDX(i)] != '\0' ){
itoa3( i - (HIST_NO-1), &preStr[1] );
tm_putstring( preStr );
tm_putstring( hist[IDX(i)] );
tm_putstring( "\n" );
}
}
}
ER commandRequest( B* cmd ) /* conversion request */
{
UW v = 0;
ER ercd;
cmd += sizeof(CMD_REQUEST) - 1; /* Skip "request". */
if( *cmd != ' ' ){
return E_PAR;
}
cmd++; /* Skip to " " */
while( isdigit(*cmd) ){ /* Strings → UW-type numbers */
v = (v*10) + (*cmd++ - '0');
}
ercd = tk_snd_mbf( mbfid, &v, sizeof(UW), TMO_POL );
return ercd; /* Ask Task X to handle it. */
}
void commandDisplay(void) /* Display the conversion result */
{
T_MSGPKT *pkt;
ER ercd;
do{ /* Receive all conversion results */
ercd = tk_rcv_mbx( mbxid, &pkt, TMO_POL );
if( ercd == E_OK ){
tm_putstring( pkt->bstr ); /* Show results */
tm_putstring( "\n" );
tk_rel_mpf( mpfid, pkt ); /* Free the memory blocks. */
}
}while( ercd == E_OK );
}
ER executeCommand( B *cmdBuf ) /* Command Execution */
{
ER ercd = E_OK;
if( !strcmp( cmdBuf, CMD_HISTORY ) ){ /* View history */
commandHistory();
}
else if( !strncmp( cmdBuf, CMD_REQUEST, sizeof(CMD_REQUEST)-1 ) ){
ercd = commandRequest( cmdBuf ); /* conversion request */
}
else if( !strcmp( cmdBuf, CMD_DISPLAY ) ){
commandDisplay(); /* Display the conversion result */
}else{
ercd = E_PAR;
tm_putstring( "NO SUCH COMMAND\n" );
}
return ercd;
}
void taskX( INT stacd, void *exinf ) /* Task X : Conversion functions */
{
T_MSGPKT *pkt; /* converted result storage packet */
UW v; /* Value to be converted */
UW uw; /* For bit processing */
B *bstr;
while(1){
tk_rcv_mbf( mbfid, &v, TMO_FEVR ); /* v: the value to be converted */
tk_get_mpf( mpfid, &pkt, TMO_FEVR ); /* For storing conversion results */
bstr = pkt->bstr;
for( uw = 0x80000000UL; uw > 0; uw >>= 1 ){
*bstr++ = (v & uw)? '1':'0'; /* binary string */
}
*bstr = '\0';
tk_snd_mbx( mbxid, (T_MSG*)pkt );
}
tk_exd_tsk(); /* I'm not coming here. */
}
void createTranslateTask(void)
{
CONST T_CMPF cmpf = { NULL, TA_TFIFO, MPFNO, MPFSZ };
CONST T_CMBF cmbf = { NULL, TA_TFIFO, 16, sizeof(UW) };
CONST T_CMBX cmbx = { NULL, TA_TFIFO|TA_MFIFO };
CONST T_CTSK ctsk = { NULL, TA_HLNG|TA_RNG0, taskX, 1, 4*1024 };
ID taskXid;
ER ercd;
/* Creating Objects */
mpfid = tk_cre_mpf( &cmpf ); /* fixed length memory pool */
mbfid = tk_cre_mbf( &cmbf ); /* message buffer */
mbxid = tk_cre_mbx( &cmbx ); /* mailbox */
taskXid = tk_cre_tsk( &ctsk ); /* task generation */
ercd = tk_sta_tsk( taskXid, 0 ); /* Launching a task */
}
EXPORT INT usermain( void )
{
CONST T_CMPL cmpl = { NULL, TA_TFIFO, MPLSZ };
B *cmdBuf;
INT i;
ER ercd;
/*--------------*/
/* 初期化 */
/*--------------*/
mplid = tk_cre_mpl( &cmpl );
for( i = 0; i < HIST_NO; i++ ){
tk_get_mpl( mplid, 1, &cmdBuf, TMO_FEVR );
*cmdBuf = '\0';
hist[i] = cmdBuf; /* Add a dummy memory block. */
}
idx = HIST_NO;
createTranslateTask(); /* Creating tasks for conversion */
/*--------------*/
/* 主ループ */
/*--------------*/
while( 1 ){
ercd = getCommand( &cmdBuf ); /* Input from the console */
if( ercd <= 0 ){ /* Errors or line breaks only */
continue;
}
if( !strcmp( cmdBuf, CMD_EXIT ) ){ /* endorsement */
break;/*while*/ /* → Escape from the loop. */
}
ercd = replaceCommand( &cmdBuf ); /* Replacement for rerun */
if( ercd < 0 ){
tm_putstring( "Replace error\n" );
continue;
}
tk_rel_mpl( mplid, hist[IDX(idx)] );
hist[IDX(idx)] = cmdBuf; /* Replacing commands */
executeCommand( cmdBuf ); /* Command Execution */
idx++; /* Next History Location */
}/*while*/
/*--------------*/
/* termination process */
/*--------------*/
for( i = 0; i < HIST_NO; i++ ){
tk_rel_mpl( mplid, hist[i] ); /* Free the memory blocks. */
}
tk_del_mpl( mplid ); /* Removing the memory pool */
return 1;
}
Listing 2 replaces the following functions that were empty in Listing 1. It also includes additional functions and definitions that are needed.
| createTranslateTask | Generate task X for conversion. |
|---|---|
| commandRequest | Processes the request command. |
| commandDisplay | Processes the display command. |
Listing 2 adds the following two commands to the list
| request | Ask Task X to convert the numbers. |
|---|---|
| Specify a number by leaving a space between them. | |
| display | Displays the result of the conversion in Task X. |
The request command converts the specified value to a 32-bit binary number, and displays the result of the conversion in another command (display).
Figure 3 shows a log of the actual execution of the command (*2).
(*2) Figure 3 shows the display when running the software in the emulator included in the T-Kernel 2.0 Software Package.
【Figure 3: Result of the execution of Listing 2.】
The actual conversion is done by another task X, which sends the value to be converted to task X using the message buffer (reference 4) to send the value to be converted to task X. The task X sets the result of the conversion to a memory block acquired from the fixed-length memory pool and sends the value to be converted to task X. Task X sets the conversion result to a memory block obtained from the fixed-length memory pool and sends a pointer to the memory block in which the conversion result is set to a mailbox (see #4) to the initial task using a mailbox.
The sequence of actions by the request and display commands is illustrated in Figure 4.
【Figure 4: Overall structure of the conversion features added in Listing 2】
It is possible to convert in the original task if it is a process of Task X in List 2.
However, if, for example, complex and time-consuming processing is required, or if it is necessary to wait for input from another task before conversion, or if it is necessary to communicate with other devices, it may be better to separate the tasks.
If the command input and the actual processing are separated, as shown in Figure 4, it is possible to perform time-consuming processing in the background while still being able to handle input from the console.
Summary
For embedded devices, the key to development is how efficiently limited resources can be used. Memory, in particular, is a resource that greatly affects power consumption and product cost, so it needs to be used as efficiently as possible.
T-Kernel provides variable-length memory pools and fixed-length memory pools as a function to use the memory efficiently. Please use these functions to develop a system for efficient memory management.
“もっと見る” カテゴリーなし
Mbed TLS overview and features
In this article, I'd like to discuss Mbed TLS, which I've touched on a few times in the past, Transport …
What is an “IoT device development platform”?
I started using Mbed because I wanted a microcontroller board that could connect natively to the Internet. At that time, …
Mbed OS overview and features
In this article, I would like to write about one of the components of Arm Mbed, and probably the most …