[ art / civ / cult / cyb / diy / drg / feels / layer / lit / λ / q / r / sci / sec / tech / w / zzz ] archive provided by lainchan.jp

lainchan archive - /λ/ - 7062



File: 1435373031905.png (498.22 KB, 300x169, tumblr_mxhwchMD6k1skyf48o1_500.gif)

No.7062

So, I've been working on developing my own i686 kernel, and I thought I would see if anyone else has had a go at operating system development. If not, who is insane enough to want to try?

My main issue at the moment is getting interrupts, well, working. Fun stuff! I would like to see what the rest of /λ/ thinks.

  No.7063

>>7062
That's very interesting. What are you writing this in?
FORTH?
C?
Assembler language?
Something else?

  No.7064

>>7063
I'm writing it in a mix of C and NASM style assembler.

It's quite fun, NASMs macros make a few things much simpler than they would be, like interrupt handlers for instance.

  No.7066

Can you share the source?

  No.7068

File: 1435416799788.png (9.45 KB, 200x116, QEMU_001.png)

>>7066
If people are really that interested I could, I would like to get it to a proper state of completion though. But if /λ/ wants to see it sooner than I can do that.

(Image Related)

  No.7139

>>7062
What kind of kernel is it? Monolithic? Modoular? Micro? Exo?

  No.7144

>>7139
It is planned to be a monolithic kernel, at least initially.

  No.7145

File: 1435556607717.png (16.99 KB, 200x118, telos.png)

I started mine a couple years ago after finishing an operating systems course. It has a really basic, read-only ext2 implementation (and a read-write tmpfs-like FS as well), a bunch of POSIX stuff like signals, timers, etc. and I recently ported binutils+gcc+newlib so I have a proper user space dev environment now. My next goal is to get enough of POSIX implemented to port a real shell.

  No.8243

Bump

I would like to see more on this topic

  No.8251

I've been interested in the front-end of OS development for a while. Bump.

  No.8263

My current project is a simple OS.
The thing about it is that it won't use the usual C ABI, but something a bit different.
bump.
as a side discussion, would you agree that M$ windex is an evolutionary dead end in OS design?

  No.8264

>>7062
What specific problem are you having OP?

  No.8272

>>7145
Ha, nice, The Talos Principle OS :P

  No.8838

>>8263
>would you agree that M$ windex is an evolutionary dead end in OS design?
Yes. And I would also agree that POSIX is a dead end too. I never understood why so many people who start creating their OSs try to implement POSIX. That's been done. If we want to innovate we need to step away from that. So many of the POSIX designs are extremely crusty. Android does some things more modernly than POSIX (hell, even Windows does); Such as running each program as its own user so as to isolate programs from each other. One can set this up, but most distros don't, for instance, run the web browser as its own user which belongs to the "internet" group (along with only the programs that can connect to the net). The principal of least privilege has been around for decades, but few if anyone is using it in such a unilateral fashion. Imagine if every program could run in a BSD Jail. If the OS was built from the outset with a permission / capability system rather than every terminal having a full suite of commands available then one wouldn't have to suffer them breaking and configuration headaches when you take the commands away. Programs could specify which commands / features they need at install time (see: groups), but /usr/bin was never designed to be used that way.

Also Consider STDIN / STDOUT / STDERR. Here we have "standard" streams of unstructured data, typically of the textual variety. Windows drops these in favor of other IPC functionality. However, even AmigaOS had a more rich IPC functionality built in where programs could attach to "ports" on each others memory spaces directly. Android has "Intents" which is an interface to standardize interprocess communications and give rich structured IO and RPC between applications. A next gen OS could take all these ideas and implement them down at the lower level (rather than up in a VM). Rather than the C mindfuckery of not knowing how big an int is, the language could instead provide standardization via explicit type sizes (like uint32_t, etc) and byte orders such that IPC could occur transparently across a network. That same IPC mechanism could also be used in the OS threading model instead of the rather cumbersome pthreads single result system. My point is: POSIX is ancient.

Furthermore, there are some great design ideas such as in MULTICS which are being "rediscovered" such as how to build out distributed storage and computing, processing as a service. The MULTICS system could hotswap in nodes, to upgrade one took a chunk of nodes offline, built the new system, then rejoined the rest to it. Unix is literally a "joke", named to thumbed ones nose at the "over-engineered" MULTICS system... meanwhile people are trying to shoehorn POSIX into the design of MULTICS without even realizing it.

Those who don't understand MULTICS will emulate it, poorly. Don't believe me? Just look at IoT (Internet of Things) -- on POSIX? All of my why?

Not that it matters, just gets old hearing people rag on Windows when *nix isn't really any better of a design. Both designs are monolithic legacy crap OSs not even built for the way we use tech today. I should be able to stop right here, push a button and continue typing on my tablet where I left off. All of my idle devices should be able to give the device I'm using cycles. But noooo, that would be too much like MULTICS.

  No.8839

>>8838
Yes

> I should be able to stop right here, push a button and continue typing on my tablet where I left off

http://www.loper-os.org/

  No.8841

>>8838
Look into Common Lisp. A Common Lisp process starts off with several standard streams open, which can be used for querying the user, sending Lisp objects elsewhere, network services, etc.

UNIX is not only a horrible design, but it has worked to actively destroy several interesting aspects of computing. We have RISC because it was discovered that C compilers didn't use most of the instructions given to them. Now hardware gets increasingly less intelligent to bend to UNIX better, even though UNIX still can't take good advantage of it.

We can also thank UNIX for contributing to the death of most operating systems and alternative hardware that used to exist.

Have you ever read the UNIX Hater's Handbook? It goes into more detail and is a great read. Funnily enough, UNIX zealots haven't changed much over the decades.

  No.8856

File: 1440712647737-0.png (7.38 KB, 200x150, samuraiBL.png)

File: 1440712647737-1.png (18.13 KB, 200x150, samuraiBLservices.png)

>>7062
I've been writing this shell~mod for an old machine I bought, so it's spanning a modern pc while being writable from floppy. Check out more here v i r t u a l m e c h dot i n f o

  No.8858

>>8856
oh you're that guy from /cyber/, hello!!

  No.8859

>>8856
Those colors makes my eyes bleed....

>samuraiBL.png


I'm not gonna lie, I thought `Boys Love` at first.

Other than that soykaf color scheme, it looks nifty, what's it written in? (because lain knows I'm not going to your site)

  No.8863

>>8838
> I should be able to stop right here, push a button and continue typing on my tablet where I left off
>>8839
> http://www.loper-os.org/

You can do that with Skullcode, including:
> idle devices should be able to give the device I'm using cycles.

Accessing a node with a browser should detect the HTTP and emit a VM in HTML/JS with a clone of the current environment that's running on the remote .exe. For the the tablet it would probably be better to install the .apk then point it at your nodes.

  No.8869

>>8858
ayy
>>8859
T5100 Orange Pulp

  No.8880

>>8838
> However, even AmigaOS had a more rich IPC functionality built in where programs could attach to "ports" on each others memory spaces directly.

It appears some FreeBSD folks agree:
http://www.nextbsd.org/jordan-hubbard-visits-bafug/

> The talk also briefly covers why mach ports are extremely useful in some cases and no UNIX IPC primitive is an adequate substitute.


What's old is new again!

  No.8888

>>8880
>What's old is new again!
Rediscovering older systems and research to discuss current inadequacies is a UNIX tradition.

  No.8891

https://news.ycombinator.com/item?id=10138447

So what is a better (in terms of design) alternative to Unix and Windows?

Minix? HURD? Some Lisp Machine from Stanislav? And even then, x86 is very complex, and has warts too. What will replace x86?

  No.8895

>>8891
>Minix
dead
>HURD
soykaf and gnu infested
>What will replace x86?
ARM, not that the hardware designers aren't doing their best to make it just as crusty as x86 as fast as possible

everything is shit, you just have to pick the soykaf that stinks the least

  No.8896

>>8891
I don't feel like reading the ycombinator link.

>So what is a better (in terms of design) alternative to Unix and Windows?

Well, there's a plethora of now dead systems to look at. There are also new systems being developed, such as templeos.
There are still people making simple Forth environments.

>Minix? HURD?

Both UNIX. HURD may be nice once it's finished. So much time and money goes into stupid soykaf like docker to poorly tack functionality onto the Linux kernel. You would get better encapsulation with HURD, which works at the kernel level and can run itself recursively.
Despite this, you have people talking about docker like it's some great innovative milestone. From what I've seen, it's stale shit.

>Some Lisp Machine from Stanislav?

You can still buy older Lisp machines. A Lisp machine would be nice.

>And even then, x86 is very complex, and has warts too. What will replace x86?

MIPS, ARM, and RISC-V are decent contenders.

You're forgetting calculators:
http://www.loper-os.org/?p=384

HP calculators were elegant little machines.

  No.8969

>>8856
This thing looks sickk!

  No.9230

File: 1441683309215.png (52.37 KB, 135x200, os3ep.jpg)

Want more inspiration?

http://pages.cs.wisc.edu/~remzi/OSTEP/

Operating Systems: Three Easy Pieces

  No.9241

>>8895
> everything is shit, you just have to pick the soykaf that stinks most alike you own shit, so your nose would adapt faster.
ftfy

  No.9507

For those looking to start playing with OS dev, I started reading this and it looks good so far for beginner.

http://littleosbook.github.io

  No.9618

>>9507
I've read that, it's okay as far as beginner books go, thanks for sharing.

  No.10579

File: 1444968384006.png (21.07 KB, 200x150, 1435892680484.jpg)

I'm currently reading Modern Operating Systems by Tanenbaum and it's been really fun. Kinda makes me want to read more about this when I finish it.

  No.11416

I've been working on my os for a little over a month and have reached the point that I am ready to make a mmaped heap for allocation.

I have decided to use a linked list of bins. These bins will hold the free memory for the system and be ordered by sizes. (i.e. bin1=1B-4B, bin2=5-8B, etc...) These bins will consist of a doublylinked linst of the free memory in address increasing order. I am wondering if this sounds sane or even plausible. Any thoughts?

  No.11418

>>10579
This stuff is excellent
http://www.projectoberon.com/
I disliked the way that Tanenbaum handwaves early on in the book making OS's with languages that have GC

  No.13035

We need more OS discussion.

  No.13036

is there any list with all the features an OS has to have? ex:
memory management, process management, interfacing with hardware, etc etc?

  No.13038

It's only tangentially related, but I've been looking at http://www.contiki-os.org/ internals, because its relatively small compared to behemoth OS's like linux.

https://www.youtube.com/watch?v=kZ-UBrVncKc

  No.13064

>>11416
I'm a little confused by what you mean by "a doubly linked list of the free memory in address increasing order". Why would that be any different from a normal fixed-size array?

  No.13066

File: 1451423812134.png (88.77 KB, 200x179, me9LJ2k.jpg)

any of yall heard of intermezzOS or redox?

  No.13068

>>13066
>memes from r/FULLCOMMUNISM

hail, comrade!

  No.13069

File: 1451424882063.png (28.92 KB, 200x200, 1535087_906687309420560_4833269303273103855_n.jpg)

>>13068
more of a r/COMPLETEANARCHY kinda guy tbh fam

  No.13070

>>13066
back on topic, intermezzOS and redox are both hobby/learning tool OS kernels written completely in Rust, which is pretty damn cool

not sure if it's quite the appropriate thread, but would zinc.rs belong here as well?

  No.13071

>>13070
I thought Rust couldn't be run without C++ bootstrap code?

  No.13073

>>11416
http://g.oswego.edu/dl/html/malloc.html

is it feasible to write an OS in a language that compiles to C? like gambit scheme?

  No.13077

>>13071
I don't think that's the case...

I think that's an LLVM dependency?

  No.13093

>>13073
that right there is one nifty algorithm.

  No.13108

>>13073
If it can interface with C, you can write an OS with it (although you'll also need at least a little bit of C and ASM to glue it together).

  No.13111

>>13073
Yes:
http://linuxfinances.info/info/lisposes.html
http://picolisp.com/wiki/?PilOS

You don't need C for an operating system though. You'll only need the assembler language of your platform for certain things:

https://common-lisp.net/project/movitz/

  No.13116

File: 1451520160951.png (1.7 MB, 141x141, savage high five.webm)

>>13108
>although you'll also need at least a little bit of C and ASM to glue it together
I was expecting that, but at least the bulk of the code should be a Lisp.
I'd prefer CL but I doubt I could make it work.

>>13111
thanks for the links mate

  No.13193

>>8838
>I never understood why so many people who start creating their OSs try to implement POSIX. That's been done.
So much this. I am planning my own OS project too and while I love Linux/UNIX-like systems I plan to make my OS something totally new, perhaps it will be better, perhaps worse but at least be unique and try something different. I think the main reason people want to implement POSIX is just so they can port over things like glxgears, gcc and vim and it all looks very impressive in screenshots.

  No.13202

>grep plan9
>nothing
:/

  No.15131

I think if I make an operating system, I'd like to focus on being able to export programs rather than import them. That way it might attract application developers, since programs written for it could be easily made cross-platform.

As a weirder idea, I'd like to make a hardware system where the OS proper is built into the hardware/firmware, possibly as a microcontroller that runs asynchronously with the user programs processor. The idea here would be to put process management on a read-only area of the system, so that it can't be overwritten by software.

Also: I have The Elements of Computing Systems: Building a Modern Computer from First Principles. As a paper book, getting it was annoying in that it has sections that tell you to go through a lesson without telling you that not only are you expected to get it from its website, but that nearly the entire book is there to begin with anyway. Dropped it after a while because it got so boring to plod through it's early lessons on basic binary operations. I'll probably get back to it sometime, but doubt that it'll be my favorite book on OS design once I've read more than one.

  No.15136

>>15131
>making OS part of the hardware
I'm not sure if that would be such a smart move. Fixing bugs would become impossible without users having to pay for new hardware. It's also kinda against the ideas of open source and hackability.
What would one actually benefit from the OS being part of the hardware? Performance?

  No.15140

>>8888
Checked

  No.15149

>>15136
The initial idea I had was that I'd like for a way to ahve the OS stay responsive no matter how badly a program running under it hangs, so I can kill the offending process without rebooting the whole system. Since I like to think of things in terms of hardware, I imagined having a secondary screen and buttons that just show the proccess tree and manipulate it even if the rest of the system is frozen. From there, I figured I would also want the lower level parts of the system to be harder to overwrite remotely, to prevent malware from stuffing code into them. So I considered having a setup where that was just impossible, by having them run on something like a Harvard architecture system with the code being read-only. Then I figured I would want to update it sometimes. But for a personal machine, I don't mind messing with the hardware a little to update it every once in a while.

So I imagined the system's main board having two very similar slots for processor-type chips. One would lack pins for altering the code, and actually run is as the process controller. The other would allow writing but wouldn't actually have control of the system, so to change the OS on a low level you would need to swap the two chips. This would be a pain to manage in an office with rows of comptuers set up all over the place, but I figure opening up the box every few months would be reasonable enough in a personal setting. Keep in mind that a lot of OS componets are technically userland, and the chip would mostly just run elements that are usually considered BIOS or kernel-level, which generally doesn't need to be changed as often. An alternate version of the system that allows writing to the process management chips would be possible, but isn't what I expect myself to want.

I don't think that would do much for general performance by itself, but I'm also interested in trying to build a system around a DSP as the main processor, for fast interrupt response time, which would fit nicely with it. On the other hand, if I go the virtual machine route I could just have the whole thing run on the audio or graphics card.

  No.15159

>>15149
>a way to ahve the OS stay responsive no matter how badly a program running under it hangs

I want to build a system like this too however I come to very different conclusions than you about how its done. I would have my OS code locked in protected memory and have strict limits on how much resources applications can use. The OS logic would always have higher priority than any userspace application with no exceptions meaning if resources are getting scarce userspace will always be hit, no matter what the kernel and system UI should be fully responsive. I think the system UI should have presidence over any userspace app, for example if this was windows the start button should always be responsive no matter what happens to programs running.

  No.15167

>>15159
Yeah, that sounds more like what I want to do with the virtual machine system, by having everything run in a language that's designed to enforce resource limitations on programs written in it. So much of what I want out of OS design I would actually get out of language design, and running it on special hardware would be mostly just having a hardware bytecode interpeter for the language.

I'd also like the process manager window to organise processes in a visual process tree like the filesystem tree, though that's something that could get tacked onto existing OSes as as a user interface tool.

  No.15168

>>15167
Do you mean like how htop and ProcessHacker do?

  No.15176

>>15168
Yeah, just like that.

  No.15391

>>15167
>>15168

So uh, quick aside, you guys know the term '''Library[-defined] OS'''?

https://www.youtube.com/watch?v=uXt4a_46qZ0

More or less be able to have your native language run both the high and low level. This is pretty dangerous, so choose your native language carefully because you want your invariants to hold.

http://fun-discoveries.blogspot.se/2016/03/compilation-as-typed-edsl-to-edsl.html

The above link is more of a suggestion: To enforce separation of concerns, have multiple domain languages that hold separate concerns. One which holds machine primitives, a second to hold resource concerns, and a third to hold capabilities. The composition of these three domains would become your OS.

In the case of the above, they used a natrual transformation between two example EDSL's to create a compile path.

  No.15845

>>13035
bump for this

  No.16631

>>15167
try doing it the other way round

  No.16634

>>7063
What's /lam/'s deal with Forth?
I've looked into it before and it honestly doesn't look that great.
The most I've seen anyone do with it is the creator who made some colored squares appear on screen.

  No.16638

>>9507
It seems interesting, but I'm trying to follow it and it's a fucking nightmare. It's full of typos, mixed up variables (background and foreground framebuffer colors are swapped for example).

I don't know if the guy who wrote it is some famous computer genius, but the "book" is written in a very amateur and lazy fashion.

  No.16656

>>16634
>I've looked into it before and it honestly doesn't look that great
Did you got past wikipedia article? You would understand how awesome and sensible Forth is if you knew more about it.

  No.16660

>>16656
Forth is great for embedded systems, but way I see it it's a slightly more advanced form of assembly.

  No.16666

>>16634
>/lam/'s
it is literally a single poster.

  No.17348

I've been working on a kernel that can run on the beagle bone black. So far I've got virtual memory working, forking with preemptive processes, and some basic ipc stuff with pipes.

If anybody is interested it is here:

https://github.com/mytchel/bom

Features, ports and criticism welcome.

  No.17351

>>17348
What books have you read? Any suggestions to what to look for in order to learn about this stuff?

  No.17353

>>17351
I skimmed through a couple Tanenbaum ones. Mostly Modern Operating Systems which coverers a lot of the more abstract side. Operating System Design by Douglas Comer is also pretty good, it follows though the complete creation of a very simple os.

Reading through the code for other projects is helpful. Fucking plan9 is god send. And data sheets of the hardware you're targetting are pretty useful

  No.17354

I am starting work on an ARM microkernel in rust. I have found wiki.osdev.org very useful.

Also I have not read any technical papers yet but if anyone is looking for a shortcut then you might want to look at rumpkernel.org (run bits of Net-BSD like filesystem drivers and the network stack in the userspace of your OS)

  No.17564

>>17354
Why'd you choose rust? Did you want to do it in something with type saftey and so on?

Do you find it makes it easier or more annoying to develop?

  No.17641

>>17348
Well. Virtual filesystems are mostly functional with a in ram filesystem working.

Now to make a real one.

  No.17936

>>8838
Would Plan 9 be similar to what you want? You seem to have a similar mentally to what Rob Pike wants in his computing.

  No.20211

File: 1479210810044.png (1.82 MB, 200x150, deal-with-it.gif)

Well fuarrrk it. I've been slacking off studying and friends for the past five odd months working on this damn project. So I may as well share it again.

https://github.com/mytchel/bom

https://lackname.org/blog/A%20Slightly%20More%20Technical%20Post%20About%20Bom/

Its a micro kernel that currently only runs on the beagle bone black. Supports fat 16/32 (should support 32, haven't actually tested) on the emmc and sd cards. Can execute files. Has a shell. Pre-emptive, processes, each with their own virtual memory.

I'm working on improving the shell and doing some write up's until I can find a job. And hopefully the job can distract me from this fuarrrking time sink.

  No.20218

you should make it so it uses only your own proprietary protacols so it wont be compatible with anything and only clinicly insane people will use it. ya know make an OS thats a joke/challenge like the mindfuck language.

  No.20219

>>20218
Someone already wrote Windows Vista.

  No.20227

I'm making a kernel myself, all processes are going to run lisp, and IPC will just be putting arbitrary objects in thread-safe queues.

Serious question: would it be a terrible idea to write a kernel in Forth?

  No.20230

>>20227
>Serious question: would it be a terrible idea to write a kernel in Forth?
Yes, because that's probably solving more problems than it needs to.

Forth works best when targeting the smallest problem that can be derived and nothing else.

  No.20231

Writing a JIT-ing lisp interpreter that runs kernel-less on the beaglebone black. Going to use StarterWare to do this.

  No.20242

>>20231
StarterWare looks interesting, just as interesting as downloading it apears to be.

How are you doing you jit inerpreter and how do you plan on interefacing it with starterware?

  No.20244

>>20242
well, I can't find any good ARM remote assemblers, so I'm writing the assembly ahead of time and the interpreter will splice it together.

starterware will mostly do the stuff that's not worth doing with lisp, like do the first part of the interrupt handler and booting and setting up the MMU and such. I plan on putting in special beaglebone-only instructions that allow the programmer to set the interrupt handlers and such.

  No.20320

>2nd year uni guy in way over his head here
I'm really interested in making my own little OS as a project, and was wondering what i need to know and good material. I know there's some good material itt, but dunno what would be best. I know C and some MIPS assembly, and just started looking at 6502 assembly. MY eventual goal is to make my own system that uses 6502 with the cpu westell makes, and make the os for it, but i don't know how within grasp that is for now. Should i just drop 6502 and learn x86 to get into os design? Or should I learn arm and ditch mips and 6502? Either way i want to build a system and makes its OS.

  No.20324

fuarrrk guys; I'm trying to get GDT and the IDT working and I just can't make it. For the last 3 days I've tried 3 completely different tutorials and I never got any to work.

I just want to get interrupts from the keyboard, anyone have a good tutorial or at least solid documentation? I'm using x86 btw.

>>20320
http://wiki.osdev.org/ is the best resource I've found so it's the one I recommend (it seems to focus on x86 mostly though), good luck.

  No.20338

>>20324
do you have a PS/2 keyboard?

USB keyboard don't send interrupts.

  No.20349

>>20338
>USB keyboard don't send interrupts.
I'm actually using a laptop. My problem is actually with loading the GDT and the IDT not with receiving the interrupt itself (haven't reached that point cuz you know, IDT not working).
But thanks for the info.

  No.20350

>>20320
You're probably better off with something simple like the 6502 than x86 if you're going to build your own system. But it still wont be simple.

  No.20351

>>20350
Unfortunately I don't have access to pretty much any equipment except for my laptop so a 6502 system would have to run fully on emulation.
What I do have is time (and hopefully, patience) so I don't mind taking a harder path.

  No.20353

>>20351
welp, I just got the GDT working, just had to disable a multiboot option in the linker.ld.

Now to get the IDT working!

  No.20453

>>20351
Bruh why are you replying to a guy that's replying to me?

  No.20454

>>20453
welp, fuarrrk me, sorry

  No.20543

Is it possible to build my cross compiler on a usb so i can go between working on my laptop and desktop? laptop has no space left for it :/

  No.20544

>>20543
along with having all of my os dev tools on that same usb

  No.20545

>>20544
I don't see why you couldn't mount the USB, download the sources into it and then compile inside it too.
However if you're really this crammed for space you should probably look for some real solutions...

  No.20546

Somewhat OT but virtual machines are an easy project to get started and can turn into full virtualized environments. Plus writing a compiler for your bizarro-assembly is good practice.

  No.20557

>>20545
Well they're both windows machines, and the laptop has a 128gb ssd that i have crammed full of school stuff

  No.20568

>>20545
It would be incredibly slow though. If you've got enough ram you could try have it build in a tmpfs then copy it to the usb.

  No.21320

I think this thread is the closest subject match for my question: over the course of reading various discussions on HN, I've become curious about some of the alternative operating systems that have (more-or-less) died out over the years: things like AmigaOS, BeOS, Plan9, Genera, etc.

Does anyone know a good resource for learning more about these systems? I'm mostly interested in the design of the systems, instead of, like, Genera For Dummies.

  No.21329

>>21320
the webpages of many of those projects will often have very in-depth discussions of how the OS is different from other OSes. That and osdev.org is how I learned most of what I know about operating systems.

  No.21370

>>10579
>Tanenbaum
Are you reading "The Dinosaur Book"? :DDDDDDDD

I think critical reading for OS design is that book and the MINIX book. If you spend a month or so tinkering with MINIX (Implementing memory-fit algorithms and processor schedulers), you would definately then be able to tinker with Linux and other Open Unix systems.