Eigenes OS?

[PHPnerd]

You see that task_switch() will never be called? you declare a variable, does 1 time incrmeent, and checks for 200?

///////////

Ok, good.

Try the same, but than in the assbly part itself, each time one line, until the GPF occurs. then we know which opcode makes the fault. (Maybe it is just the jump).

If that fails, try getting all values of the registers when GPF triggers (print out all registers etc from the Registers given at the handler and post them, and maybe a disassembly of your schedule() (so we can exactly see when it trigger according to RIP's. We cant put cli hlt etc into the assembly part

// PHPnerd

[Erhard Henkes]

PHPnerd schrieb:

You see that task_switch() will never be called? you declare a variable, does 1 time incrmeent, and checks for 200?

c is declared static, therefore it works:
..

[PHPnerd]

What is the info from the #GP?

// PHP

[Erhard Henkes]

Currently, I do not get GPF again. I have done the following:

1. I used the clobber list:

asm volatile("..." : : "r"(eip), "r"(esp), "r"(ebp), "r"(current_directory->physicalAddr) : "ebx", "edx");

ebx and edx were accepted, edi and esi gave failure ("can't find a register in class `GENERAL_REGS' while reloading `asm'").
2) I accelerated the task switch:

//TEST
    static ULONG c = 0;
    ++c;
    if(c>0)
    {
      printformat("task_switch\n");
      task_switch();
      settextcolor(getpid(),0);
      c=0;
    }
    //TEST

output at bochs:

!task_switch
!task_switch
!task_switch
!HEAP start: 40081000h end: 40100000h max: 4FFFF000h kernel mode: 0 read-only: 0

ask_switch
!
hole 40081000h hole-size: 0007F000h
after create_heap: placement_address: 0030C020h allocated frames: 795
ask_switch
!
task_switch
!tasking install
!tasking install
after moving stack, ESP: 0018FFD0h ddress: 0030C020h
After k_mallocing kernel_stack), placement_address: 0030C020h kernel_stack: 4010
1000h
VFS & RAM Disk install
ask_switch
!!!!#!!!!#!!!!##
placement_address after ram disk install: 40100000h
ramdisk_start: 40081000h file_data_start: 0000A2F0h file_data_end: 0000B662h
ask_switch
!Page Fault (page not present) at 00D2ECF0h - EIP: 0000D2C9h
Page Fault >>> Exception. System Halted! <<<

As you can see, there is one failure free task switch.
But the output looks disturbed.

What is the info from the #GP?

Ihave no specific info from GPF. You mentioned a failure analysis routine. Do you have a link for that code or a table with the error flag meanings?

I post the current version with the problematic task switch in the time handler:
http://www.henkessoft.de/OS_Dev/Downloads/37.zip

I use gcc.exe (version 3.1) and ld.exe (version 2.13) due to linking aout-format of NASM
http://www.osdever.net/downloads/compilers/DJGPP-Installer-nocpp.exe

[PHPnerd]

Well done!

Just, remove the ! and # and task switch printouts

Do you have a keyboard handler? Try to make a simple console system. Use the Function keys to switch to another console. Each task is linked to a console. When switching console, you copy the data from 0xb8000 to the data of current console. And write data of new console to 0xb8000.

Something like that will work.

Then, you will see on each console the output of each task apart nice for testing your multitasker.

About the GP: The code of JamesM tutorials uses the structure registers. Filled in some assebly code. Use that structure to printout eip, esp etc, and get the control registers and print them out too. Error flag meanings can be found in the Intel Manual Volume 1.

// PHPnerd

[Erhard Henkes]

Try to make a simple console system. Use the Function keys to switch to another console. Each task is linked to a console.

This sounds good. I have a keyboard handler and an own circular key queue. A console in your concept is an array of 4000 Bytes which is swapped with the video RAM by the task switcher or by function keys? Not bad.

Well done!

Now I have exchanged the GPF for Page Faults:

paging install
frames: 00300000h NFRAMES: 524288
kernel_directory->physicalAddr: 00306000h
placement_address: 0030B000h allocated frames: 795
HEAP start: 40081000h end: 40100000h max: 4FFFF000h kernel mode: 0 read-only: 0
hole start: 40081000h end: 40100000h max: 4FFFF000h kernel mode: 0 read-only: 0
hole 40081000h hole-size: 0007F000h
after create_heap: placement_address: 0030C020h allocated frames: 795
tasking install
after moving stack, ESP: 0018FFD0h
After init first task (kernel task), placement_address: 0030C020h
After k_mallocing kernel_stack), placement_address: 0030C020h kernel_stack: 4010
1000h
VFS & RAM Disk install
placement_address after ram disk install: 40200000h
ramdisk_start: 40081000h file_data_start: 0000A2E0h file_data_end: 0000B652h
After set_kernel_stack ==> tss_entry.esp0: 40101800h

> IRQ 127 <<<
Hello, user world!
Found file dev
(directory)
Found file f1
contents: "PrettyOS: My filename is test1.txt!"
age Fault (page not present) at 0F000123h - EIP: 0F000123h
Page Fault >>> Exception. System Halted! <<<

Dependent on the value of c in the timer handler the page fault comes earlier or later. With c=2000 (means 20 sec) it came with the fisrt task switch:

ksldfgjskldfgjldfksjgklsdfgklsdfjgklsdfjglksdfjglksdfjgsklgjklsd;jglkdsfjglkdsfg
jlsk;dfjgskldfjgslkdjglk;sdgjldk;sfjglk;sdfjgl;ksdjgkl;sdgjklsdfgjsl;dkjglsdk;fg
js 0000D794h H: 0000D794h WRITE: 156 Read: 156 *T: g 103 *H: g 103
T: 0000D78Eh H: 0000D78Eh WRITE: 162 Read: 162 *T: f 102 *H: f 102
Page Fault (page not present) at 0040FC20h - EIP: 0040FC20h
Page Fault >>> Exception. System Halted! <<<
...

Hence, I think the EIP gets a wrong value under certain circumstances.
The function read_eip() is defined in Assembler (process.asm):

global _read_eip
_read_eip:
    pop eax               ; Get the return address
    jmp eax               ; Return. Can't use RET because return
                          ; address popped off the stack.

About the GP: The code of JamesM tutorials uses the structure registers. Filled in some assebly code. Use that structure to printout eip, esp etc, and get the control registers and print them out too. Error flag meanings can be found in the Intel Manual Volume 1.

Thanks, great idea.

[PHPnerd]

read_eip() is fine. But you could try:

readEip:
    pop eax
    push eax
    ret

You are having some drivers, right?

When switching tasks, you change the registers, and the cr3. When a task does something with a device, and there is not good code implemented to handle multitasking in device drivers, the device will move data. But when in that time a task switch occurs, some problems may come. The task switch at 20seconds is very bad. The process doesn't need to know it is EVER switched. 100Hz may be nice (100 switches each second). Play something with it.

Consoles: you got it

typedef struct
{
    uint32_t id; // console id, just for making some things easier
    uint8_t keystroke[KEYSTROKE_LENGTH]; // make this the way you did with keystrokes
    uint32_t *videoBuffer; // a pointer to some place in memory, size: width*height*2 bytes (probably 80*25*2)
    uint32_t cursorX;
    uint32_t cursorY; // coordinates of the cursor for this console
    uint8_t colorAttribute; // attributes curently used by console
} console_t

struct task
{
    ....
    console_t *console; // put console in task
    ....
};

That is an example for a console structure. Be sure of:
- Create one console when initializing screen (VGA), set it into a global variable, currentConsole
- Be sure the functions work to the right console. Each time a task switch occurs, it should use another console structure. But not the currentConsole, because that is the console currently SHOWED. En when not watching task 2, it should be able to print text, so the user can see it later.

You can determine the console of the current task by just using currentTask->console(->cursorX, ->cursorY etc.)

It is simple, but much work. Also be sure to make a backup from all you have first, it is a lot of work to remove the console code from your VGA driver.

Good luck, I think it will work with this.

// PHPnerd

[Erhard Henkes]

global _read_eip
_read_eip: 
    pop eax
    push eax
    ret

... looks better and works fine, but does not help with the page fault due to the strange EIP.

Thanks for the consoles' code. Nice feature.

[PHPnerd]

Does the page fault trigger after setting eip, or after setting cr3? or something else?

// PHPnerd

[Erhard Henkes]

Ich habe mit HLT im asm-code probiert, der page fault erfolgt beim *jmp %%ecx;
Was bedeutet eigentlich dieser * vor %%ecx ? Wenn ich ihn weglasse, passiert genau das gleiche.
Ich habe eip,esp,ebp,cur_dir->phys direkt vor dem asm-code geloggt:
..
Schalten nach 0,01 sec:

task_switch before asm: eip: 0000D306h esp: 0018FF3Ch ebp: 0018FF54h cur_dir->phys: 00001000h
Page Fault (page not present) at 0F000123h - EIP: 0F000123h

Schalten nach 1 sec: (dann wurden zumindest die neuen Tasks angelegt)

task_switch before asm: eip: 0000D4C2h esp: 0018FFD8h ebp: 0018FFF0h cur_dir->phys: 0041E000h
Page Fault (page not present) at 0040FCA0h - EIP: 0040FCA0h

Wenn ich das richtig sehe, liegt das Problem eher bei current_directory->physicalAddr als bei eip, oder?.

Im Lowlevel Forum meinte man dazu, dass das Problem vom C-Compiler kommen könnte (nicht alle register wurden gespeichert und zurück geschrieben), und man empfahl mir das komplett in Assembler zu machen analog tyndur.

Ohne die Clobber List kommt übrigens wieder der GPF:

...
jmp %%ecx;           "
: : "r"(eip), "r"(esp), "r"(ebp), "r"(current_directory->physicalAddr) );

task_switch before asm: eip: 0000D4C2h esp: 0018FFD8h ebp: 0018FFF0h cur_dir->phys: 0041E000h
General Protection Fault >>> Exception. System Halted! <<<

Erst, wenn ich sowohl ebx als auch edx "clobbere", kommt der PF, einzeln GPF. edi und esi kann ich leider nicht clobbern wegen Fehlermeldung des Compilers.

Fundstelle zur Clobber List:
http://groups.google.im/group/comp.lang.asm.x86/browse_thread/thread/cf89391bb64126f7
..

Das kommt mir inzwischen alles suspekt vor, was gcc da schafft. Mit dem Clobbern habe ich wenig Erfahrung.

[abc.w]

Erhard Henkes schrieb:

Was bedeutet eigentlich dieser * vor %%ecx ?...

Siehe hier http://www.c-plusplus.net/forum/viewtopic-var-t-is-238553.html

[PHPnerd]

Did you try to get the assembly code generated by GCC?

Are you sure that the currentDir->physical is physical?
And are the new page tables, and page directories you made, page aligned?

// PHPnerd

[Erhard Henkes]

Did you try to get the assembly code generated by GCC?

Yes, I was interested in the consequence of putting (or not putting) registers into the clobber list. Here is the result:

clobbering ebx and edx ==> Page Fault:

asm volatile("         \
      cli;                 \
      mov %0, %%ecx;       \
      mov %1, %%esp;       \
      mov %2, %%ebp;       \
      mov %3, %%cr3;       \
      mov $0x12345, %%eax; \
      sti;                 \
      jmp %%ecx;           "
      : : "r"(eip), "r"(esp), "r"(ebp), "r"(current_directory->physicalAddr) 
      : "ebx","edx" );

objdump task.o -D > task.txt

35f:	fa                   	cli    
 360:	89 f1                	mov    %esi,%ecx
 362:	89 fc                	mov    %edi,%esp
 364:	89 cd                	mov    %ecx,%ebp
 366:	0f 22 d8             	mov    %eax,%cr3
 369:	b8 45 23 01 00       	mov    $0x12345,%eax
 36e:	fb                   	sti    
 36f:	ff e1                	jmp    *%ecx

The first thing you see is why we cannot clobber edi and esi, too.

Empty clobber list ==> GPF

351:	fa                   	cli    
 352:	89 d9                	mov    %ebx,%ecx
 354:	89 f4                	mov    %esi,%esp
 356:	89 fd                	mov    %edi,%ebp
 358:	0f 22 d8             	mov    %eax,%cr3
 35b:	b8 45 23 01 00       	mov    $0x12345,%eax
 360:	fb                   	sti    
 361:	ff e1                	jmp    *%ecx

clobbering ebx ==> GPF

35f:	fa                   	cli    
 360:	89 f1                	mov    %esi,%ecx
 362:	89 fc                	mov    %edi,%esp
 364:	89 d5                	mov    %edx,%ebp
 366:	0f 22 d8             	mov    %eax,%cr3
 369:	b8 45 23 01 00       	mov    $0x12345,%eax
 36e:	fb                   	sti    
 36f:	ff e1                	jmp    *%ecx

clobbering edx ==> GPF

351:	fa                   	cli    
 352:	89 d9                	mov    %ebx,%ecx
 354:	89 f4                	mov    %esi,%esp
 356:	89 fd                	mov    %edi,%ebp
 358:	0f 22 d8             	mov    %eax,%cr3
 35b:	b8 45 23 01 00       	mov    $0x12345,%eax
 360:	fb                   	sti    
 361:	ff e1                	jmp    *%ecx

[Erhard Henkes]

..

[Erhard Henkes]

Ich habe nun noch eine Variable für die physikalische Adresse in fork() eingefügt, um diese dort auszugeben:

// Create a new process.
    ULONG phys;
    task_t* new_task = (task_t*) k_malloc( sizeof(task_t),0,&phys );
    printformat("placement_address after creating a new process: %x phys: %x\n", placement_address, phys);

VFS & RAM Disk install
placement_address after ram disk install: 40200000h
ramdisk_start: 40081000h file_data_start: 0000A2E0h file_data_end: 0000B652h
After set_kernel_stack ==> tss_entry.esp0: 40101800h

> IRQ 127 <<<
Hello, user world!
...
placement_address after creating a new process: 40200000h phys: 0041CBE8h
fork() returned: 00000002h and getpid() returned: 00000001h
placement_address after creating a new process: 40200000h phys: 0041CC18h
fork() returned: 00000003h and getpid() returned: 00000001h
placement_address after creating a new process: 40200000h phys: 0041CC48h
fork() returned: 00000004h and getpid() returned: 00000001h
SS: 00000010h, ESP: 0018FFE0h, EBP: 0018FFF0h, CS: 00000018h, DS: 00000020h
task_switch before asm: eip: 0000D4CFh esp: 0018FFD8h ebp: 0018FFF0h cur_dir->ph
ys: 0041E000h
Page Fault (page not present) at 0040FCC0h - EIP: 0040FCC0h

[PHPnerd]

I will explain the kmalloc for you.

size = size (doh)
align = page aligned, this makes your address page aligned, usable for paging structures and TSS (and others)
phys = get the physical address. You can put NULL in it, or a pointer to a variable. Like:

uint32_t phys = 0;
uint32_t ptr = kmallocAP(0x100, 1, &phys);
printf("Phys: 0x%X, Virt: 0x%X\n",phys,ptr);

Getting physical address and doing nothing with it won't solve your problem.
The task doesn't have to be page aligned, since it is Software Multitasking, and you made it your own (nah, JamesM did :P)

The kmalloc code is 100% fine. (I used it my own some time ago)

// PHPnerd

[Erhard Henkes]

Yes, the k_malloc(...) of James Molloy (I have his written permission to use his source code as a base for own experiments in an OS tutorial, if I like, but I am not sure, if his code is really robust enough) is usable and is not the problem (Other people think, JM has put too much into paging.h/paging.c, because phys./virt. memory management should be separated clearly. This can be done easily.).

OK, alignment is not important. I was not quite sure about it.

Currently, I do not understand the difference in the result of EIP between cases 1) and 3). In 3) I just save the physical address at 0x01 or a variable phys. The reason must be in the way of saving details of the tasks. Hence, the clobbering can influence the deficient process.

I think, I should go back to the test code where the task switch really works and analyze the effects by looking at all the details again. I just did and the page fault came, too. Thus, the failure has another reason. That's good.
As soon as I have analyzed it, I will come back.

@PHPnerd: Thanks for staying with me in times of confusion.

I use the following "stable version":
ckernel.c (like main.c):

UCHAR c=0;
	while(TRUE)
	{

	  if( k_checkKQ_and_print_char() )
	  {
	    ++c;

	    if(c>5)
	    {
	      c=0;
	      settextcolor(4,0);
	      printformat("\nT: %x H: %x WRITE: %i Read: %i ", pODA->pTailKQ, pODA->pHeadKQ, pODA->KQ_count_write, pODA->KQ_count_read);
          printformat("*T: %c %i *H: %c %i\n", *(pODA->pTailKQ),*(pODA->pTailKQ),*(pODA->pHeadKQ),*(pODA->pHeadKQ));
          settextcolor(2,0);

          task_switch(); settextcolor(getpid(),0);
	    }
	  }
	}

kjsdfhsdkjksdjfhksdjfhsdkjfhksdjfhksdjfhsdkjfhsdkjfhksdjfhkjsdfhksjdhfß
T: 0000D80Fh H: 0000D80Fh WRITE: 66 Read: 66 *T: d 100 *H: d 100rnel_stack: 4010
task_switch before asm: eip: 0000D303h esp: 0018FFD8h ebp: 0018FFF0h cur_dir->ph
ys: 00426000hk install
placement_address after ram disk install: 40200000h
ramdisk_start: 40081000h file_data_start: 0000A2C0h file_data_end: 0000B632h
After set_kernel_stack ==> tss_entry.esp0: 40101800h

> IRQ 127 <<<
Hello, user world!
Found file dev
(directory)
Found file f1
contents: "PrettyOS: My filename is test1.txt!"
Found file f2
contents: "PrettyOS: My filename is test2.txt!"
Found file f3
contents: "PrettyOS: My filename is test3.txt!"
placement_address after creating a new process: 40200000h phys: 0041CBE8h
fork() returned: 00000002h and getpid() returned: 00000001h
placement_address after creating a new process: 40200000h phys: 0041CC18h
fork() returned: 00000003h and getpid() returned: 00000001h
placement_address after creating a new process: 40200000h phys: 0041CC48h
fork() returned: 00000004h and getpid() returned: 00000001h
SS: 00000010h, ESP: 0018FFE0h, EBP: 0018FFF0h, CS: 00000018h, DS: 00000020h

While entering keystrokes, the task is switched after every sixth keystroke.
The logs of 'task_switch before asm:' with the original and 3 forked tasks are:

task_switch before asm: eip: 0000D4AFh esp: 0018FFD8h ebp: 0018FFF0h cur_dir->phys: 0041E000h //PID 2

task_switch before asm: eip: 0000D4AFh esp: 0018FFD8h ebp: 0018FFF0h cur_dir->phys: 00422000h //PID 3

task_switch before asm: eip: 0000D4AFh esp: 0018FFD8h ebp: 0018FFF0h cur_dir->phys: 00426000h //PID 4

task_switch before asm: eip: 0000D303h esp: 0018FFD8h ebp: 0018FFF0h cur_dir->phys: 00001000h //PID 1

task_switch before asm: eip: 0000D303h esp: 0018FFD8h ebp: 0018FFF0h cur_dir->phys: 0041E000h //PID 2

task_switch before asm: eip: 0000D303h esp: 0018FFD8h ebp: 0018FFF0h cur_dir->phys: 00422000h //PID 3

task_switch before asm: eip: 0000D303h esp: 0018FFD8h ebp: 0018FFF0h cur_dir->phys: 00426000h //PID 4

task_switch before asm: eip: 0000D303h esp: 0018FFD8h ebp: 0018FFF0h cur_dir->phys: 00001000h //PID 1 - the same as line 4

ff.

These are the data of the forked tasks:

phys: 0041CBE8h
fork() returned: 00000002h

phys: 0041CC18h
fork() returned: 00000003h

phys: 0041CC48h
fork() returned: 00000004h

SS: 00000010h, ESP: 0018FFE0h, EBP: 0018FFF0h, CS: 00000018h, DS: 00000020h

The task switch runs "unclobbered". Now it comes:

If I clobber ebx and edx (see above), then I get a PF:

task_switch before asm: eip: 0000D4AFh esp: 0018FFD8h ebp: 0018FFF0h cur_dir->phys: 0041E000h
Page Fault (page not present) at 0040FCA0h - EIP: 0040FCA0h

If I shift the task switch to timer handler, then I get an "Absturz":

task_switch before asm: eip: 0000D303h esp: 0018FE74h ebp: 0018FE8Ch cur_dir->phys: 00001000h
...
task_switch before asm: eip: 0000D303h esp: 0018FE74h ebp: 0018FE8Ch cur_dir->phys: 00001000h
...
task_switch before asm: eip: 0000D303h esp: 0018FF70h ebp: 0018FF88h cur_dir->phys: 00001000h
...
task_switch before asm: eip: 0000D303h esp: 0018FE74h ebp: 0018FE8Ch cur_dir->phys: 00001000h
...
task_switch before asm: eip: 0000D4AFh esp: 0018FFD8h ebp: 0018FFF0h cur_dir->phys: 0041E000h
...
placement_address after creating a new process: 40200000h phys: 0041CC18h
fork() returned: 00000003h and getpid() returned: 00000002h
placement_address after creating a new process: 40200000h phys: 0041CC48h
fork() returned: 00000004h and getpid() returned: 00000002h
SS: 00000010h, ESP: 0018FFE0h, EBP: 0018FFF0h, CS: 00000008h, DS: 00000010h

==> Absturz (no exception, no reboot, just nothing besides blinking cursor)

I think, this shows that JM's function task_switch() does not correctly work. Caring only for eip, esp, ebp seems to be not enough! Right?

[Erhard Henkes]

@PHPnerd: How do you switch tasks? Are you willing to show your new kmalloc?

[PHPnerd]

I do not switch tasks yet. I will explain main for it later here

And, I am using DogLea Algorithm with my own code.
So:
Headers and footers
List containing free blocks, sorted by size

Like the JamesM one, but than all my own code (like it more :P)

Maybe look deeper in Intel Manuals of the source of other OSes to get idea's

// PHPnerd

[PHPnerd]

With the multitasker:

Try to save more registers, since you don't have a TSS.

Save:
EFLAGS <--
Segment registers: CS, DS, ES, FS, GS
Possible save some general registers (can be done later)

Maybe the EFLAGS will save the problem.
Oh, and, did you do a end-of-interrupt before task switch when using switching in PIT? May be important.

// PHPnerd