INITIALIZATION Knowledgebase: ki-dev-large Base Query: Please summarize the whole context. It is important that you include a summary for each file. All files should be included, so please make sure to go through the entire context Model: gemini-1.5-pro-preview-0409 Use Curl?: ================================================== **Elapsed Time: 0.00 seconds** ================================================== ROUTING Query type: summary ================================================== **Elapsed Time: 2.01 seconds** ================================================== PRIMER Primer: You are Simon, a highly intelligent personal assistant in a system called KIOS. You are a chatbot that can read knowledgebases through the "CONTEXT" that is included in the user's chat message. Your role is to act as an expert at summarization and analysis. In your responses to enterprise users, prioritize clarity, trustworthiness, and appropriate formality. Be honest by admitting when a topic falls outside your scope of knowledge, and suggest alternative avenues for obtaining information when necessary. Make effective use of chat history to avoid redundancy and enhance response relevance, continuously adapting to integrate all necessary details in your interactions. Use as much tokens as possible to provide a detailed response. ================================================== **Elapsed Time: 0.38 seconds** ================================================== FINAL QUERY Final Query: CONTEXT: ########## File: Git%20Magic%20-%20Ben%20Lynn%2C%20et%20al.%20%28HTML%2C%20PDF%2C%20EPUB%29.pdf Page: 1 Context: # Git Magic **Ben Lynn** August 2007 ## Preface Git is a version control Swiss army knife. A reliable versatile multipurpose revision control tool whose extraordinary flexibility makes it tricky to learn, let alone master. As Arthur C. Clarke observed, any sufficiently advanced technology is indistinguishable from magic. This is a great way to approach Git: newbies can ignore its inner workings and view Git as a gizmo that can amaze friends and infuriate enemies with its wondrous abilities. Rather than go into details, we provide rough instructions for particular effects. After repeated use, gradually you will understand how each trick works, and how to tailor the recipes for your needs. - **Simplified Chinese:** by JunJie, Meng and JiangWei. Converted to Traditional Chinese via iconv -f UTF-8 -t UTF-8-TW. - **French:** by Alexandre Garel, Paul Gaborit, and Nicolas Deram. - **German:** by Benjamin Bellee and Armin Stebich; also hosted on Armin's website. - **Italian:** by Mattia Rigotti. - **Korean:** by Jung-Ho (John) Han; also hosted on John's website. - **Polish:** by Damian Michna. - **Brazilian Portuguese:** by José Inácio Serafini and Leonardo Siqueira Rodrigues. - **Russian:** by Tikhon Taravasky, Mikhail Dymkov, and others. - **Spanish:** by Rodrigo Toledo and Ariset Llerena Tapia. - **Ukrainian:** by Volodymyr Bodevchik. - **Vietnamese:** by Trần Ngọc Quân; also hosted on his website. - Single webpage: barebones HTML, with no CSS. #################### File: Git%20Magic%20-%20Ben%20Lynn%2C%20et%20al.%20%28HTML%2C%20PDF%2C%20EPUB%29.pdf Page: 2 Context: # Documentation - **PDF file**: printer-friendly. - **EPUB file**: E-reader-friendly. - **Debian package**: Ubuntu package; get a fast and local copy of this site. Handy when this server is offline. - **Physical book**: [Amazon](https://www.amazon.com): 64 pages, 15.24cm x 22.86cm, black and white. Handy when there is no electricity. ## Thanks! I'm humbled that so many people have worked on translations of these pages. I greatly appreciate having a wider audience because of the efforts of those named above. - Dustin Sallings - Alberto Bertogli - James Cameron - Douglas Livingston - Michael Budde - Richard Albury - Tarmigan - Derek Mahar - Frode Aanevik - Keith Patrick - Andy Somerville - Ralf Recker - Øyvind A. Holm - Miklos Vajna - Sébastien Hinderer - Thomas Miedl - Joe Malin - Tyler Breese - Sonia Hamilton - Julian Haagsman - Roman Lesniak - Sergey Litvinov - Oliver Ferrigni - David Toca - Jodi Thiefry - Baiju Muthukadan contributed corrections and improvements. François Marier maintains the Debian package originally created by Daniel Baumann. My gratitude goes to many others for your support and praise. I'm tempted to quote you here, but it might raise expectations to ridiculous heights. If I've left you out by mistake, please tell me or just send me a patch! ## License This guide is released under the GNU General Public License, version 3 or any later version published by the Free Software Foundation. Naturally, the source is kept in a Git repository and can be obtained by typing: ``` $ git clone git://repo.or.cz/gitmisc.git # Creates "gitmisc" directory. ``` or from one of the mirrors: ``` $ git clone git://github.com/blyn/gitmagic.git $ git clone git://git.asembla.com/gitmagic.git $ git clone git@bitbucket.org:blyn/gitmagic.git ``` GitHub, Asembla, and Bitbucket support private repositories; the latter two for free. #################### File: Git%20Magic%20-%20Ben%20Lynn%2C%20et%20al.%20%28HTML%2C%20PDF%2C%20EPUB%29.pdf Page: 3 Context: # Introduction I'll use an analogy to introduce version control. See the [Wikipedia entry](https://en.wikipedia.org/wiki/Version_control) on revision control for a saner explanation. ## Work is Play I've played computer games almost all my life. In contrast, I only started using version control systems as an adult. I suspect I'm not alone, and comparing the two may make these concepts easier to explain and understand. Think of editing your code, or document, as playing a game. Once you’ve made a lot of progress, you’d like to save. To do so, you click on the **Save** button in your text editor. But this will overwrite the old version. It’s like those old school games which only had one save slot: sure you could save, but you could never go back to an older state. Which was a shame, because your previous save might have been right at an exceptionally fun part of the game that you’d like to revisit one day. Or worse still, your current save is in an unwinable state, and you have to start again. ## Version Control When editing, you can **Save As...** a different file, or copy the file somewhere first before saving if you want to savor old versions. You can compress them too to save space. This is a primitive and labour-intensive form of version control. Computer games improved on this long ago, many of them providing multiple automatically timestamped save slots. Let’s make the problem slightly tougher. Say you have a bunch of files that go together, such as source code for a project, or files for a website. Now if you want to keep an old version you have to archive a whole directory. Keeping many versions around by hand is inconvenient, and quickly becomes expensive. With some computer games, a saved game really does consist of a directory full of files. These games hide this detail from the player and present a convenient interface to manage different versions of this directory. Version control systems are no different. They all have nice interfaces to manage a directory of stuff. You can save the state of the directory every so often, and you can load any one of the saved states later on. Unlike most computer games, they’re usually smart about conserving space. Typically, only a few files change from version to version, and not much. Storing the differences instead of entire new copies saves room. #################### File: Git%20Magic%20-%20Ben%20Lynn%2C%20et%20al.%20%28HTML%2C%20PDF%2C%20EPUB%29.pdf Page: 4 Context: # Distributed Control Now imagine a very difficult computer game. So difficult to finish that many experienced gamers all over the world decide to team up and share their saved games to try to beat it. Speedruns are real-life examples: players specializing in different levels of the same game collaborate to produce amazing results. How would you set up a system so they can get at each other’s saves easily? And upload new ones? In the old days, every project used centralized version control. A server somewhere held all the saved games. Nobody else did. Every player kept at most a few saved games on their machine. When a player wanted to make progress, they’d download the latest save from the main server, play a while, save and upload back to the server for everyone else to use. What if a player wanted to get an older saved game for some reason? Maybe the current saved game is in an unwinnable state because somebody forgot to pick up an object back in level three, and they want to find the latest saved game where the game can still be completed. Or maybe they want to compare two older saved games to see how much work a particular player did. There could be many reasons to want to see an older revision, but the outcome is the same. They have to ask the central server for that old saved game. The more saved games they want, the more they need to communicate. The new generation of version control systems, of which Git is a member, are known as distributed systems, and can be thought of as a generalization of centralized systems. When players download from the main server, they get every saved game, not just the latest one. It’s as if they’re mirroring the central server. This initial cloning operation can be expensive, especially if there’s a long history, but it pays off in the long run. One immediate benefit is that when an old save is desired for any reason, communication with the central server is unnecessary. ## A Silly Superstition A popular misconception is that distributed systems are ill-suited for projects requiring an official central repository. Nothing could be further from the truth. Photographing someone does not cause their soul to be stolen. Similarly, cloning the master repository does not diminish its importance. A good first approximation is that anything a centralized version control system can do, a well-designed distributed system can do better. Network resources are simply costlier than local resources. While we shall later see there are drawbacks to a distributed approach, one is less likely to make erroneous comparisons with this rule of thumb. A small project may only need a fraction of the features offered by such a system. #################### File: Git%20Magic%20-%20Ben%20Lynn%2C%20et%20al.%20%28HTML%2C%20PDF%2C%20EPUB%29.pdf Page: 5 Context: # Git Basics ## Merge Conflicts For this topic, our computer game analogy becomes too thinly stretched. Instead, let us again consider editing a document. Suppose Alice inserts a line at the beginning of a file, and Bob appends one at the end of his copy. They both upload their changes. Most systems will automatically deduce a reasonable course of action: accept and merge their changes, so both Alice's and Bob's edits are applied. Now suppose both Alice and Bob have made distinct edits to the same line. Then it is impossible to proceed without human intervention. The second person to upload is informed of a **merge conflict**, and must choose one edit over another, or revise the line entirely. More complex situations can arise. Version control systems handle the simpler cases themselves, and leave the difficult cases for humans. Usually their behavior is configurable. ## Basic Tricks Rather than diving into a sea of Git commands, use these elementary examples to get your feet wet. Despite their simplicity, each of them are useful. Indeed, in my first months with Git I never ventured beyond the material in this chapter. ### Saving State About to attempt something drastic? Before you do, take a snapshot of all files in the current directory with: ```bash $ git init $ git add . $ git commit -m "My first backup" ``` Now if your new edits go awry, restore the pristine version: ```bash $ git reset --hard ``` To save the state again: ```bash $ git commit -a -m "Another backup" ``` #################### File: Git%20Magic%20-%20Ben%20Lynn%2C%20et%20al.%20%28HTML%2C%20PDF%2C%20EPUB%29.pdf Page: 6 Context: # Add, Delete, Rename The above only keeps track of the files that were present when you first ran `git add`. If you add new files or subdirectories, you'll have to tell Git: ```bash $ git add readme.txt Documentation ``` Similarly, if you want Git to forget about certain files: ```bash $ git rm kludge.h obsolete.c $ git rm -r incriminating/evidence/ ``` `git` deletes these files for you if you haven't already. Renaming a file is the same as removing the old name and adding the new name. There's also the shortcut `git mv` which has the same syntax as the `mv` command. For example: ```bash $ git mv bug.c feature.c ``` # Advanced Undo/Redo Sometimes you just want to go back and forget about every change past a certain point because they’re all wrong. Then: ```bash $ git log ``` shows you a list of recent commits, and their SHA1 hashes: ``` commit 766f98189604240ba334153047649b88b8f11c664 Author: Bob Date: Tue Mar 14 01:59:26 2000 -0800 Replace printf() with write(). commit 82f5ea3462e651544956a8653c05f58d151275c Author: Alice Date: Thu Jan 1 00:00:00 1970 +0000 Initial commit. ``` The first few characters of the hash are enough to specify the commit; alternatively, copy and paste the entire hash. Type: ```bash $ git reset --hard 766f ``` to restore the state to a given commit and erase all newer commits from the record permanently. Other times you want to hop to an old state briefly. In this case, type: ```bash $ git checkout 8215 ``` #################### File: Git%20Magic%20-%20Ben%20Lynn%2C%20et%20al.%20%28HTML%2C%20PDF%2C%20EPUB%29.pdf Page: 7 Context: This takes you back in time, while preserving server commits. However, like time travel in a science-fiction movie, if you now edit and commit, you will be in an alternate reality, because your actions are different to what they were the first time around. This alternate reality is called a **branch**, and we’ll have more to say about this later. For now, just remember that: ```bash $ git checkout master ``` will take you back to the present. Also, to stop Git complaining, always commit or reset your changes before running checkout. To take the computer game analogy again: - `git reset --hard`: load an old save and delete all saved games newer than the one just loaded. - `git checkout`: load an old game, but if you play on, the game state will deviate from the newer saves you made the first time around. Any saved games you make now will end up in a separate branch representing the alternate reality you have entered. We deal with this later. You can choose only to restore particular files and subdirectories by appending them after the command: ```bash $ git checkout some.file another.file ``` Take care, as this form of `checkout` can silently overwrite files. To prevent accidents, commit before running any checkout command, especially when first learning Git. In general, whenever you feel unsure about any operation, Git command or not, first run: ```bash $ git commit -a ``` Don’t like cutting and pasting hashes? Then use: ```bash $ git checkout :/"My first b" ``` to jump to the commit that starts with a given message. You can also ask for the 5th-last saved state: ```bash $ git checkout master~5 ``` ## Reverting In a court of law, events can be stricken from the record. Likewise, you can pick specific commits to undo. ```bash $ git commit -a $ git revert ``` will undo just the commit with the given hash. The revert is recorded as a new commit, which you can confirm by running: ```bash $ git log ``` #################### File: Git%20Magic%20-%20Ben%20Lynn%2C%20et%20al.%20%28HTML%2C%20PDF%2C%20EPUB%29.pdf Page: 8 Context: # Changelog Generation Some projects require a changelog. Generate one by typing: ```bash $ git log > ChangeLog ``` ## Downloading Files Get a copy of a project managed with Git by typing: ```bash $ git clone git://server/path/to/files ``` For example, to get all the files I used to create this site: ```bash $ git clone git://git.or.cz/gitmagic.git ``` We'll have much to say about the clone command soon. ## The Bleeding Edge If you've already downloaded a copy of a project using `git clone`, you can upgrade to the latest version with: ```bash $ git pull ``` ## Instant Publishing Suppose you've written a script you'd like to share with others. You could just tell them to download from your computer, but if they do so while you're improving the script or making experimental changes, they could wind up in trouble. Of course, this is why release cycles exist. Developers may work on a project frequently, but they only make the code available when they feel it is presentable. To do this with Git, in the directory where your script resides: ```bash $ git init $ git add . $ git commit -m "First release" ``` Then tell your users to run: ```bash $ git clone your.computer:/path/to/script ``` to download your script. This assumes they have SSH access. If not, run `git daemon` and tell your users to instead run: ```bash $ git clone git://your.computer/path/to/script ``` From now on, every time your script is ready for release, execute: ```bash $ git commit -a -m "Next release" ``` #################### File: Git%20Magic%20-%20Ben%20Lynn%2C%20et%20al.%20%28HTML%2C%20PDF%2C%20EPUB%29.pdf Page: 9 Context: and your users can upgrade their version by changing to the directory containing your script and typing: ```bash $ git pull ``` Your users will never end up with a version of your script you don’t want them to see. ## What Have I Done? Find out what changes you’ve made since the last commit with: ```bash $ git diff ``` Or since yesterday: ```bash $ git diff "@{yesterday}" ``` Or between a particular version and 2 versions ago: ```bash $ git diff 1b6d "master-2" ``` In each case, the output is a patch that can be applied with `git apply`. Try also: ```bash $ git whatchanged --since="2 weeks ago" ``` Often I’ll browse history with `git` instead, due to its slick photogenic interface, or `tig`, a text-mode interface that works well over slow connections. Alternatively, install a web server, run `git instantweb` and fire up any web browser. ## Exercise Let A, B, C, D be four successive commits where B is the same as A except some files have been removed. We want to add the files back at D. How can we do this? There are at least three solutions. Assuming we are at D: 1. The difference between A and B are the removed files. We can create a patch representing this difference and apply it: ```bash $ git diff B A | git apply ``` 2. Since we saved the files back at A, we can retrieve them: ```bash $ git checkout A -- foo.c bar.h ``` 3. We can view going from A to B as a change we want to undo: ```bash $ git revert B ``` Which choice is best? Whichever you prefer most. It is easy to get what you want with Git, and often there are many ways to get it. #################### File: Git%20Magic%20-%20Ben%20Lynn%2C%20et%20al.%20%28HTML%2C%20PDF%2C%20EPUB%29.pdf Page: 10 Context: # Cloning Around In older version control systems, checkout is the standard operation to get files. You retrieve a bunch of files in a particular saved state. In Git and other distributed version control systems, cloning is the standard operation. To get files, you create a clone of the entire repository. In other words, you practically mirror the central server. Anything the main repository can do, you can do. ## Sync Computers I can tolerate making tarballs or using `rsync` for backups and basic syncing. But sometimes I edit on my laptop, other times on my desktop, and the two may not have talked to each other in between. 1. Initialize a Git repository and commit your files on one machine. Then on the other: ``` $ git clone other.computer:/path/to/files ``` To create a second copy of the files and Git repository. From now on, run: ``` $ git commit -a $ git pull other.computer:/path/to/files HEAD ``` This will pull in the state of the files on the other computer into the one you’re working on. If you’ve recently made conflicting edits in the same file, Git will let you know and you should commit again after resolving them. ## Classic Source Control Initialize a Git repository for your files: ``` $ git init $ git add . $ git commit -m "Initial commit" ``` On the central server, initialize a bare repository in some directory: ``` $ mkdir proj.git $ cd proj.git $ git --bare init $ touch proj.git/git-daemon-export-ok ``` Start the Git daemon if necessary: ``` $ git daemon --detach # it may already be running ``` For Git hosting services, follow the instructions to set up the initially empty Git repository. Typically one fills in a form on a webpage. Push your project to the central server with: ``` # Command to push instructions (not provided in the original text) ``` #################### File: Git%20Magic%20-%20Ben%20Lynn%2C%20et%20al.%20%28HTML%2C%20PDF%2C%20EPUB%29.pdf Page: 11 Context: ``` $ git push central.server/path/to/proj.git HEAD To check out the source, a developer types: ```bash $ git clone central.server/path/to/proj.git ``` After making changes, the developer saves changes locally: ```bash $ git commit -a ``` To update to the latest version: ```bash $ git pull ``` Any merge conflicts should be resolved then committed: ```bash $ git commit -a ``` To check in local changes into the central repository: ```bash $ git push ``` If the main server has new changes due to activity by other developers, the push fails, and the developer should pull the latest version, resolve any merge conflicts, then try again. Developers must have SSH access for the above pull and push commands. However, anyone can see the source by typing: ```bash $ git clone git://central.server/path/to/proj.git ``` The native Git protocol is like HTTP: there is no authentication, so anyone can retrieve the project. Accordingly, by default, pushing is forbidden via the Git protocol. ## Secret Source For a closed-source project, omit the touch command, and ensure you never create a file named `git-daemon-export-ok`. The repository can no longer be retrieved via the Git protocol; only those with SSH access can see it. If all your repos are closed, running the Git daemon is unnecessary because all communication occurs via SSH. ## Bare Repositories A bare repository is so named because it has no working directory: it only contains files that are normally hidden away in the `.git` subdirectory. In other words, it maintains the history of a project and never holds a snapshot of any given version. A bare repository plays a role similar to that of the main server in a centralized version control system: the home of your project. Developers clone your project from it and push the latest official changes to it. Typically, it resides on a server. ``` #################### File: Git%20Magic%20-%20Ben%20Lynn%2C%20et%20al.%20%28HTML%2C%20PDF%2C%20EPUB%29.pdf Page: 13 Context: # Light-Speed Multitask Say you want to work on several features in parallel. Then commit your project and run: ``` git clone ./some/new/directory ``` Thanks to hardlinking, local clones require less time and space than a plain backup. You can now work on two independent features simultaneously. For example, you can edit one clone while the other is compiling. At any time, you can commit and pull changes from the other clone: ``` git pull /the/other/clone HEAD ``` # Guerilla Version Control Are you working on a project that uses some other version control system, and you sorely miss Git? Then initialize a Git repository in your working directory: ``` git init git add . git commit -m "Initial commit" ``` Then clone it: ``` git clone ./some/new/directory ``` Now go to the new directory and work here instead, using Git to your heart's content. Once in a while, you'll want to sync with everyone else, in which case go to the original directory, sync using the other version control system, and type: ``` git add . git commit -m "Sync with everyone else" ``` Then go to the new directory and run: ``` git commit -a -m "Description of my changes" git pull ``` The procedure for giving your changes to everyone else depends on the other version control system. The new directory contains the files with your changes. Run whatever commands of the other version control system are needed to upload them to the central repository. Subversion, perhaps the best centralized version control system, is used by countless projects. The `git svn` command automates the above for Subversion repositories and can also be used to export a Git project to a Subversion repository. #################### File: Git%20Magic%20-%20Ben%20Lynn%2C%20et%20al.%20%28HTML%2C%20PDF%2C%20EPUB%29.pdf Page: 14 Context: # Mercurial Mercurial is a similar version control system that can almost seamlessly work in tandem with Git. With the `hg-git` plugin, a Mercurial user can losslessly push to and pull from a Git repository. ## Obtain the `hg-git` Plugin with Git: ```bash $ git clone git://github.com/schacon/hg-git.git ``` or Mercurial: ```bash $ hg clone http://bitbucket.org/durin42/hg-git/ ``` Sadly, I am unaware of an analogous plugin for Git. For this reason, I advocate Git over Mercurial for the main repository, even if you prefer Mercurial. With a Mercurial project, usually a volunteer maintains a parallel Git repository to accommodate Git users, whereas thanks to the `hg-git` plugin, a Git project automatically accommodates Mercurial users. Although the plugin can convert a Mercurial repository to a Git repository by pushing to an empty repository, this job is easier with the `hg-fast-export.sh` script, available from: ```bash $ git clone git://repo.or.cz/fast-export.git ``` To convert, in an empty directory: ```bash $ git init $ hg-fast-export.sh -r /hg/repo ``` After adding the script to your `$PATH`. # Bazaar We briefly mention Bazaar because it is the most popular free distributed version control system after Git and Mercurial. Bazaar has the advantage of hindsight, as it is relatively young; its designers could learn from mistakes of the past, and sidestep minor historical warts. Additionally, its developers are mindful of portability and interoperation with other version control systems. A `bzr-git` plugin lets Bazaar users work with Git repositories to some extent. The tailor program converts Bazaar repositories to Git repositories, and can do so incrementally, while `bzr-fast-export` is well-suited for one-shot conversions. # Why I Use Git I originally chose Git because I heard it could manage the unimaginably unmanageable Linux kernel source. I've never felt a need to switch. Git has served #################### File: Git%20Magic%20-%20Ben%20Lynn%2C%20et%20al.%20%28HTML%2C%20PDF%2C%20EPUB%29.pdf Page: 15 Context: # Branch Wizardry Instant branching and merging are the most lethal of Git's killer features. ## Problem External factors inevitably necessitate context switching. A severe bug manifests in the released version without warning. The deadline for a certain feature is moved closer. A developer whose help you need for a key section of the project is about to leave. In all cases, you must abruptly drop what you are doing and focus on a completely different task. Interrupting your train of thought can be detrimental to your productivity, and the more cumbersome it is to switch contexts, the greater the loss. With centralized version control, we must download a fresh working copy from the central server. Distributed systems fare better, as we can clone the desired version locally. But cloning still entails copying the whole working directory as well as the entire history up to the given point. Even though Git reduces the cost of this with file sharing and hard links, the project files themselves must be recreated in their entirety in the new working directory. ## Solution Git has a better tool for these situations that is much faster and more space-efficient than cloning: `git branch`. With this magic word, the files in your directory suddenly shapeshift from one version to another. This transformation can do more than merely go back or forward in history. Your files can morph from the last release to the experimental version to the current development version to your friend’s version and so on. # The Boss Key Ever played one of those games where at the push of a button (“the boss key”), the screen would instantly display a spreadsheet or something? So if the boss walked into the office while you were playing the game you could quickly hide it away? #################### File: Git%20Magic%20-%20Ben%20Lynn%2C%20et%20al.%20%28HTML%2C%20PDF%2C%20EPUB%29.pdf Page: 16 Context: In some directory: ```bash $ echo "I'm smarter than my boss" > myfile.txt $ git init $ git add . $ git commit -m "Initial commit" ``` We have created a Git repository that tracks one text file containing a certain message. Now type: ```bash $ git checkout -b boss # nothing seems to change after this $ echo "My boss is smarter than me" > myfile.txt $ git commit -a -m "Another commit" ``` It looks like we’ve just overwritten our file and committed it. But it’s an illusion. Type: ```bash $ git checkout master # switch to original version of the file ``` And hey presto! The text file is restored. And if the boss decides to snoop around this directory, type: ```bash $ git checkout boss # switch to version suitable for boss' eyes ``` You can switch between the two versions of the file as much as you like, and commit to each independently. ### Dirty Work Say you're working on some feature, and for some reason, you need to go back three versions and temporarily put in a few print statements to see how something works. Then: ```bash $ git commit -a $ git checkout HEAD~3 ``` Now you can add ugly temporary code all over the place. You can even commit these changes. When you're done, type: ```bash $ git checkout master ``` to return to your original work. Observe that any uncommitted changes are carried over. What if you wanted to save the temporary changes after all? Easy: ```bash $ git checkout -b dirty ``` And commit before switching back to the master branch. Whenever you want to return to the dirty changes, simply type: ```bash $ git checkout dirty ``` #################### File: Git%20Magic%20-%20Ben%20Lynn%2C%20et%20al.%20%28HTML%2C%20PDF%2C%20EPUB%29.pdf Page: 17 Context: We touched upon this command in an earlier chapter, when discussing loading old states. At last we can tell the whole story: the files change to the requested state, but we must leave the master branch. Any commits made from now on can take your files down a different road, which can be named later. In other words, after checking out an old state, Git automatically puts you in a new, unnamed branch, which can be named and saved with `git checkout -b`. ## Quick Fixes You’re in the middle of something when you are told to drop everything and fix a newly discovered bug in commit 186d...: ```bash $ git commit -a $ git checkout -b fixes 186d ``` Then once you’ve fixed the bug: ```bash $ git commit -a -m "Bug fixed" $ git checkout master ``` And resume work on your original task. You can even merge in the freshly baked bugfix: ```bash $ git merge fixes ``` ## Merging With some version control systems, creating branches is easy but merging them back together is tough. With Git, merging is so trivial that you might be unaware of it happening. We actually encountered merging long ago. The `pull` command in fact fetches commits and then merges them into your current branch. If you have no local changes, then the merge is a fast forward, a degenerate case akin to fetching the latest version in a centralized version control system. But if you do have local changes, Git will automatically merge and report any conflicts. Ordinarily, a commit has exactly one parent commit, namely, the previous commit. Merging branches together produces a commit with at least two parents. This begs the question: what commit does `HEAD~1` really refer to? A commit could have multiple parents, so which one do we follow? It turns out this notation chooses the first parent every time. This is desirable because the current branch becomes the first parent during a merge; frequently you’re only concerned with the changes you made in the current branch, as opposed to changes merged in from other branches. You can refer to a specific parent with a caret. For example, to show the logs from the second parent: #################### File: Git%20Magic%20-%20Ben%20Lynn%2C%20et%20al.%20%28HTML%2C%20PDF%2C%20EPUB%29.pdf Page: 18 Context: ``` $ git log HEAD~2 You may omit the number for the first parent. For example, to show the differences with the first parent: $ git diff HEAD~ You can combine this notation with other types. For example: $ git checkout 16bd~2-10 -b ancient starts a new branch “ancient” representing the state 10 commits back from the second parent of the first parent of the commit starting with 16bd. ## Uninterrupted Workflow Often in hardware projects, the second step of a plan must await the completion of the first step. A car undergoing repairs might sit idly in a garage until a particular part arrives from the factory. A prototype might wait for a chip to be fabricated before construction can continue. Software projects can be similar. The second part of a new feature may have to wait until the first part has been released and tested. Some projects require your code to be reviewed before accepting it, so you might wait until the first part is approved before starting the second part. Thanks to painless branching and merging, we can bend the rules and work on Part II before Part I is officially ready. Suppose you have committed Part I and sent it for review. Let’s say you’re in the `master` branch. Then branch off: $ git checkout -b part2 Next, work on Part II, committing your changes along the way. To err is human, and often you’ll want to go back and fix something in Part I. If you’re lucky, or very good, you can skip these lines: ```bash $ git checkout master # Go back to Part I. $ fix_problem $ git commit -a # Commit the fixes. $ git checkout part2 # Go back to Part II. $ git merge master # Merge in those fixes. ``` Eventually, Part I is approved: ```bash $ git checkout master # Go back to Part I. $ submit files # Release to the world! $ git merge part2 # Merge in Part II. $ git branch -d part2 # Delete "part2" branch. ``` Now you’re in the `master` branch again, with Part II in the working directory. ``` #################### File: Git%20Magic%20-%20Ben%20Lynn%2C%20et%20al.%20%28HTML%2C%20PDF%2C%20EPUB%29.pdf Page: 19 Context: It's easy to extend this trick for any number of parts. It's also easy to branch off retroactively; suppose you belatedly realize you should have created a branch 7 commits ago. Then type: ```bash git branch -m master part2 # Rename "master" branch to "part2". git branch master HEAD~7 # Create new "master," 7 commits upstream. ``` The master branch now contains just Part I, and the part2 branch contains the rest. We are in the latter branch; we created master without switching to it, because we want to continue work on part2. This is unusual. Until now, we've been switching to branches immediately after creation, as in: ```bash git checkout HEAD~7 -b master # Create a branch, and switch to it. ``` ## Reorganizing a Medley Perhaps you like to work on all aspects of a project in the same branch. You want to keep works-in-progress to yourself and want others to see your commits only when they have been neatly organized. Start a couple of branches: ```bash git branch sanitized # Create a branch for sanitized commits. git checkout -b medley # Create and switch to a branch to work in. ``` Next, work on anything: fix bugs, add features, add temporary code, and so forth, committing often along the way. Then: ```bash git checkout sanitized # Switch back to the sanitized branch. git cherry-pick medley # Apply commits from medley to sanitized. ``` This applies the grandparent of the head commit of the "medley" branch to the "sanitized" branch. With appropriate cherry-picks, you can construct a branch that contains only permanent code and has related commits grouped together. ## Managing Branches List all branches by typing: ```bash git branch ``` By default, you start in a branch named "master." Some advocate leaving the "master" branch untouched and creating new branches for your own edits. The `-d` and `-m` options allow you to delete and move (rename) branches. See `git help branch`. The "master" branch is a useful custom. Others may assume that your repository has a branch with this name and that it contains the official version of your project. Although you can rename or obliterate the "master" branch, you might as well respect this convention. #################### File: Git%20Magic%20-%20Ben%20Lynn%2C%20et%20al.%20%28HTML%2C%20PDF%2C%20EPUB%29.pdf Page: 20 Context: # Temporary Branches After a while, you may realize you are creating short-lived branches frequently for similar reasons: every other branch merely serves to save the current state so you can briefly hop back to an older state to fix a high-priority bug or something. It’s analogous to changing the TV channel temporarily to see what else is on. But instead of pushing a couple of buttons, you have to create, check out, merge, and delete temporary branches. Luckily, Git has a shortcut that is as convenient as a TV remote control: ``` $ git stash ``` This saves the current state in a temporary location (a stash) and restores the previous state. Your working directory appears exactly as it was before you started editing, and you can fix bugs, pull in upstream changes, and so on. When you want to go back to the stashed state, type: ``` $ git stash apply # You may need to resolve some conflicts. ``` You can have multiple stashes and manipulate them in various ways. See `git help stash`. As you may have guessed, Git maintains branches behind the scenes to perform this magic trick. # Work How You Want You might wonder if branches are worth the bother. After all, `clones` are almost as fast, and you can switch between them with `cd` instead of esoteric Git commands. Consider web browsers. Why support multiple tabs as well as multiple windows? Because allowing both accommodates a wide variety of styles. Some users like to keep only one browser window open, and use tabs for multiple webpages. Others might insist on the other extreme: multiple windows with no tabs anywhere. Others still prefer something in between. Branching is like tabs for your working directory, and cloning is like opening a new browser window. These operations are fast and local, so why not experiment to find the combination that best suits you? Git lets you work exactly how you want. # Lessons of History A consequence of Git’s distributed nature is that history can be edited easily. But if you tamper with the past, take care: only rewrite that part of history which you alone possess. Just as nations forever argue over who committed what atrocity, if someone else has a clone whose version of history differs to yours, you will have trouble reconciling when your trees interact. #################### File: Git%20Magic%20-%20Ben%20Lynn%2C%20et%20al.%20%28HTML%2C%20PDF%2C%20EPUB%29.pdf Page: 21 Context: Some developers strongly feel history should be immutable, warts and all. Others feel trees should be made presentable before they are unleashed in public. Git accommodates both viewpoints. Like cloning, branching, and merging, rewriting history is simply another power Git gives you. It is up to you to use it wisely. # I Stand Corrected Did you just commit, but wish you had typed a different message? Then run: ``` $ git commit --amend ``` to change the last message. Realized you forgot to add a file? Run `git add` to add it, and then run the above command. Want to include a few more edits in that last commit? Then make those edits and run: ``` $ git commit --amend -a ``` ## ... And Then Some Suppose the previous problem is ten times worse. After a lengthy session you've made a bunch of commits. But you're not quite happy with the way they're organized, and some of those commit messages could use rewording. Then type: ``` $ git rebase -i HEAD~10 ``` and the last 10 commits will appear in your favourite `$EDITOR`. A sample excerpt: ``` pick 56be673 Added repo.or.cz link pick a311a64 Reordered analogies in "Work How You Want" pick 100834f Added push target to Makefile ``` Older commits precede newer commits in this list, unlike the log command. Here, `56be673` is the oldest commit, and `100834f` is the newest. Then: - Remove commits by deleting lines. Like the revert command, but off the record: it will be as if the commit never existed. - Reorder commits by reordering lines. Replace `pick` with: - `edit` to mark a commit for amending. - `reword` to change the log message. - `squash` to merge a commit with the previous one. - `fixup` to merge a commit with the previous one and discard the log message. #################### File: Git%20Magic%20-%20Ben%20Lynn%2C%20et%20al.%20%28HTML%2C%20PDF%2C%20EPUB%29.pdf Page: 22 Context: For example, we might replace the second pick with squash: ``` pick 5c6b673 Added repo.or.cz link squash a311a6a Reordered analogies in "Work How You Want" pick 100384f Added push target to Makefile ``` After we save and quit, Git merges a311a6a into 5c6b673. Thus, squash merges into the next commit up: think “squash up”. Git then combines their log messages and presents them for editing. The command `fixup` skips this step; the squashed log message is simply discarded. If you marked a commit with `edit`, Git returns you to the past, to the oldest such commit. You can amend the old commit as described in the previous section, and even create new commits that belong here. Once you’re pleased with the “recton”, go forward in time by running: ```bash $ git rebase --continue ``` Git replays commits until the next edit, or to the present if none remain. You can also abandon the rebase with: ```bash $ git rebase --abort ``` So commit early and commit often: you can tidy up later with rebase. ## Local Changes Last You’re working on an active project. You make some local commits over time, and then you sync with the official tree in a merge. This cycle repeats itself a few times before you’re ready to push to the central tree. But now the history in your local Git clone is a messy jumble of your changes and the official changes. You’d prefer to see all your changes in one contiguous section, and after all the official changes. This is a job for `git rebase` as described above. In many cases you can use the `--onto` flag and avoid interaction. Also see `git help rebase` for detailed examples of this amazing command. You can split commits. You can even rearrange branches of a tree. Take care: `rebase` is a powerful command. For complicated rebases, first make a backup with `git clone`. ## Rewriting History Occasionally, you need the source control equivalent of airbrushing people out of official photos, erasing them from history in a Stalin-esque fashion. For example, suppose we intend to release a project, but it involves a file that should be kept private for some reason. Perhaps I left my credit card number in a text file and #################### File: Git%20Magic%20-%20Ben%20Lynn%2C%20et%20al.%20%28HTML%2C%20PDF%2C%20EPUB%29.pdf Page: 23 Context: # Making History Want to migrate a project to Git? If it’s managed with one of the more well-known systems, then chances are someone has already written a script to export the whole history to Git. Otherwise, look up `git fast-import`, which reads text input in a specific format to create Git history from scratch. Typically a script using this command is hastily cobbled together and run once, migrating the project in a single shot. As an example, paste the following listing into a temporary file, such as `/tmp/history`: ``` commit refs/heads/master committer Alice Thu, 01 Jan 1970 00:00:00 +0000 data < int main() { printf("Hello, World!\\n"); return 0; } EOT commit refs/heads/master committer Bob Tue, 14 Mar 2000 01:59:26 -0800 data < #include int main() { write(1, "Hello, world!\n", 14); return 0; } ``` Then create a Git repository from this temporary file by typing: ```bash $ mkdir project; cd project; git init $ git fast-import --date-format=rfc2822 < /tmp/history ``` You can checkout the latest version of the project with: ```bash $ git checkout master ``` The `git fast-export` command converts any repository to the `git fast-import` format, whose output you can study for writing exporters, and also to transport repositories in a human-readable format. Indeed, these commands can send repositories of text files over text-only channels. ## Where Did It All Go Wrong? You've just discovered a broken feature in your program which you know for sure was working a few months ago. Argh! Where did this bug come from? If only you had been testing the feature as you developed. It's too late for that now. However, provided you've been committing often, Git can pinpoint the problem: ```bash $ git bisect start $ git bisect bad HEAD $ git bisect good b16d ``` Git checks out a state halfway in between. Test the feature, and if it’s still broken: ```bash $ git bisect bad ``` If not, replace "bad" with "good". Git again transports you to a state halfway between the known good and bad versions, narrowing down the possibilities. After a few iterations, this binary search will lead you to the commit that caused the trouble. Once you’ve finished your investigation, return to your original state by typing: ```bash $ git bisect reset ``` #################### File: Git%20Magic%20-%20Ben%20Lynn%2C%20et%20al.%20%28HTML%2C%20PDF%2C%20EPUB%29.pdf Page: 25 Context: ``` $ git bisect reset Instead of testing every change by hand, automate the search by running: $ git bisect run my_script Git uses the return value of the given command, typically a one-off script, to decide whether a change is good or bad: the command should exit with code 0 when good, 125 when the change should be skipped, and anything else between 1 and 127 if it is bad. A negative return value aborts the bisect. You can do much more: the help page explains how to visualize bisects, examine or replay the bisect log, and eliminate known innocent changes for a speedier search. ## Who Made It All Go Wrong? Like many other version control systems, Git has a blame command: $ git blame bug.c which annotates every line in the given file showing who last changed it, and when. Unlike many other version control systems, this operation works offline, reading only from local disk. ## Personal Experience In a centralized version control system, history modification is a difficult operation, and only available to administrators. Cloning, branching, and merging are impossible without network communication. So are basic operations such as browsing history, or committing a change. In some systems, users require network connectivity just to view their own changes or open a file for editing. Centralized systems preclude working offline, and need more expensive network infrastructure, especially as the number of developers grows. Most importantly, all operations are slower to some degree, usually to the point where users shun advanced commands unless absolutely necessary. In extreme cases this is true of even the most basic commands. When users must run slow commands, productivity suffers because of an interrupted work flow. I experienced these phenomena first-hand. Git was the first version control system I used. I quickly grew accustomed to it, taking many features for granted. I simply assumed other systems were similar; choosing a version control system ought to be no different from choosing a text editor or web browser. I was shocked when later forced to use a centralized system. A flaky internet connection matters little with Git, but makes development unbearable when it needs to be as reliable as local disk. Additionally, I found myself conditioned to avoid certain commands because of the latencies involved, which ultimately prevented me from following my desired work flow. ``` #################### File: Git%20Magic%20-%20Ben%20Lynn%2C%20et%20al.%20%28HTML%2C%20PDF%2C%20EPUB%29.pdf Page: 26 Context: When I had to run a slow command, the interruption to my train of thought dealt a disproportionate amount of damage. While waiting for server communication to complete, I'd do something else to pass the time, such as check email or write documentation. By the time I returned to the original task, the command had finished long ago, and I would waste more time trying to remember what I was doing. Humans are bad at context switching. There was also an interesting tragedy of the commons effect: anticipating network congestion, individuals would consume more bandwidth than necessary on various operations in an attempt to reduce future delays. The combined efforts intensified congestion, encouraging individuals to consume even more bandwidth next time to avoid even longer delays. ## Multiplayer Git Initially I used Git on a private project where I was the sole developer. Amongst the commands related to Git's distributed nature, I needed only pull and clone so I could keep the same project in different places. Later I wanted to publish my code with Git and include changes from contributors. I had to learn how to manage projects with multiple developers from all over the world. Fortunately, this is Git's forte, and arguably its raison d'être. ## Who Am I? Every commit has an author name and email, which is shown by `git log`. By default, Git uses system settings to populate these fields. To set them explicitly, type: ```bash $ git config --global user.name "John Doe" $ git config --global user.email johndoe@example.com ``` Omit the global flag to set these options only for the current repository. ## Git Over SSH, HTTP Suppose you have SSH access to a web server, but Git is not installed. Though less efficient than its native protocol, Git can communicate over HTTP. Download, compile and install Git in your account, and create a repository in your web directory: ```bash $ GIT_DIR=proj.git git init $ cd proj.git $ git --bare update-server-info $ cp hooks/post-update.sample hooks/post-update ``` For older versions of Git, the copy command fails and you should run: #################### File: Git%20Magic%20-%20Ben%20Lynn%2C%20et%20al.%20%28HTML%2C%20PDF%2C%20EPUB%29.pdf Page: 27 Context: ```markdown $ chmod +x hooks/post-update Now you can publish your latest edits via SSH from any clone: ``` $ git push web.server:/path/to/proj.git master ``` and anybody can get your project with: ``` $ git clone http://web.server/proj.git ``` ## Git Over Anything Want to synchronize repositories without servers, or even a network connection? Need to improvise during an emergency? We’ve seen `git fast-export` and `git fast-import` can convert repositories to a single file and back. We could shuttle such files back and forth to transport git repositories over any medium, but a more efficient tool is `git bundle`. ### The sender creates a bundle: ``` $ git bundle create somefile HEAD ``` then transports the bundle, `somefile`, to the other party somehow: email, thumb drive, an xxd printout and an OCR scanner, reading bits over the phone, smoke signals, etc. The receiver retrieves commits from the bundle by typing: ``` $ git pull somefile ``` The receiver can even do this from an empty repository. Despite its size, `somefile` contains the entire original git repository. In larger projects, eliminate waste by bundling only changes the other repository lacks. For example, suppose the commit `1b6d...` is the most recent commit shared by both parties: ``` $ git bundle create somefile HEAD "1b6d" ``` If done frequently, one could easily forget which commit was last sent. The help page suggests using tags to solve this. Namely, after you send a bundle, type: ``` $ git tag -f lastbundle HEAD ``` and create new refresher bundles with: ``` $ git bundle create newbundle HEAD ^lastbundle ``` ## Patches: The Global Currency Patches are text representations of your changes that can be easily understood by computers and humans alike. This gives them universal appeal. You can email a patch to developers no matter what version control system they’re using. As long as your audience can read their email, they can see your edits. Similarly, ``` #################### File: Git%20Magic%20-%20Ben%20Lynn%2C%20et%20al.%20%28HTML%2C%20PDF%2C%20EPUB%29.pdf Page: 28 Context: # Working with Patches in Git On your side, all you require is an email account: there’s no need to set up an online Git repository. ## Recall from the First Chapter: ```bash $ git diff 1b6d > my.patch ``` This outputs a patch which can be pasted into an email for discussion. In a Git repository, type: ```bash $ git apply < my.patch ``` to apply the patch. In more formal settings, when author names and perhaps signatures should be recorded, generate the corresponding patches past a certain point by typing: ```bash $ git format-patch 1b6d ``` The resulting files can be given to `git-send-email`, or sent by hand. You can also specify a range of commits: ```bash $ git format-patch 1b6d..HEAD ``` On the receiving end, save an email to a file, then type: ```bash $ git am < email.txt ``` This applies the incoming patch and also creates a commit, including information such as the author. With a browser email client, you may need to click a button to see the email in its raw original form before saving the patch to a file. There are slight differences for mbox-based email clients, but if you use one of these, you’re probably the sort of person who can figure them out easily without reading tutorials. ## Sorry, We’ve Moved After cloning a repository, running `git push` or `git pull` will automatically push to or pull from the original URL. How does Git do this? The secret lies in config options created with the clone. Let’s take a peek: ```bash $ git config --list ``` The `remote.origin.url` option controls the source URL; “origin” is a nickname given to the source repository. As with the “master” branch convention, we may change or delete this nickname but there is usually no reason for doing so. If the original repository moves, we can update the URL via: ```bash $ git config remote.origin.url git://new.url/proj.git ``` #################### File: Git%20Magic%20-%20Ben%20Lynn%2C%20et%20al.%20%28HTML%2C%20PDF%2C%20EPUB%29.pdf Page: 29 Context: The branch `master.merge` option specifies the default remote branch in a `git pull`. During the initial clone, it is set to the current branch of the source repository, so even if the HEAD of the source repository subsequently moves to a different branch, a later pull will faithfully follow the original branch. This option only applies to the repository we first cloned from, which is recorded in the option `branch.master.remote`. If we pull in from other repositories, we must explicitly state which branch we want: ```bash $ git pull git://example.com/other.git master ``` The above explains why some of our earlier push and pull examples had no arguments. ## Remote Branches When you clone a repository, you also clone all its branches. You may not have noticed this because Git hides them away; you must ask for them specifically. This prevents branches in the remote repository from interfering with your branches and also makes Git easier for beginners. List the remote branches with: ```bash $ git branch -r ``` You should see something like: ``` origin/HEAD origin/master origin/experimental ``` These represent branches and the HEAD of the remote repository and can be used in regular Git commands. For example, suppose you have made many commits and wish to compare against the last fetched version. You could search through the logs for the appropriate SHA1 hash, but it’s much easier to type: ```bash $ git diff origin/HEAD ``` Or you can see what the “experimental” branch has been up to: ```bash $ git log origin/experimental ``` ## Multiple Remotes Suppose two other developers are working on our project, and we want to keep tabs on both. We can follow more than one repository at a time with: ```bash $ git remote add other git://example.com/some_repo.git $ git pull other some_branch ``` Now we have merged in a branch from the second repository, and we have easy access to all branches of all repositories: #################### File: Git%20Magic%20-%20Ben%20Lynn%2C%20et%20al.%20%28HTML%2C%20PDF%2C%20EPUB%29.pdf Page: 30 Context: ```markdown $ git diff origin/experimental* other/some_branch-5 But what if we just want to compare their changes without affecting our own work? In other words, we want to examine their branches without having their changes invade our working directory. Then rather than pull, run: ```bash $ git fetch # Fetch from origin, the default. $ git fetch other # Fetch from the second programmer. ``` This just fetches histories. Although the working directory remains untouched, we can refer to any branch of any repository in a Git command because we now possess a local copy. Recall that behind the scenes, a pull is simply a fetch then merge. Usually we pull because we want to merge the latest commit after a fetch; this situation is a notable exception. See `git help remote` for how to remove remote repositories, ignore certain branches, and more. ## My Preferences For my projects, I like contributors to prepare repositories from which I can pull. Some Git hosting services let you host your own fork of a project with the click of a button. After I fetch a tree, I run Git commands to navigate and examine the changes, which ideally are well-organized and well-described. I merge my own changes and perhaps make further edits. Once satisfied, I push to the main repository. Though I infrequently receive contributions, I believe this approach scales well. See this [blog post by Linus Torvalds](https://example.com). Staying in the Git world is slightly more convenient than patch files, as it saves me from converting them to Git commits. Furthermore, Git handles details such as recording the author’s name and email address, as well as the time and date, and asks the author to describe their own change. ## Git Grandmastery By now, you should be able to navigate the `git help` pages and understand almost everything. However, pinpointing the exact command required to solve a given problem can be tedious. Perhaps I can save you some time; below are some recipes I have needed in the past. ## Source Releases For my projects, Git tracks exactly the files I’d like to archive and release to users. To create a tarball of the source code, I run: ```bash ``` ``` #################### File: Git%20Magic%20-%20Ben%20Lynn%2C%20et%20al.%20%28HTML%2C%20PDF%2C%20EPUB%29.pdf Page: 31 Context: ``` $ git archive --format=tar --prefix=proj-1.2.3/ HEAD # Commit What Changed Telling Git when you’ve added, deleted and renamed files is troublesome for certain projects. Instead, you can type: ```bash $ git add . $ git add -u ``` Git will look at the files in the current directory and work out the details by itself. Instead of the second add command, run `git commit -a` if you also intend to commit at this time. See `git help ignore` for how to specify files that should be ignored. You can perform the above in a single pass with: ```bash $ git ls-files -d -o -z | xargs -0 git update-index --add --remove ``` The `-z` and `-o` options prevent ill side-effects from filenames containing strange characters. As this command adds ignored files, you may want to use the `-x` or `-X` option. # My Commit Is Too Big! Have you neglected to commit for too long? Been coding furiously and forgotten about source control until now? Made a series of unrelated changes, because that’s your style? No worries. Run: ```bash $ git add -p ``` For each edit you make, Git will show you the hunk of code that was changed, and ask if it should be part of the next commit. Answer with “y” or “n”. You have other options, such as postponing the decision; type “?” to learn more. Once you’re satisfied, type: ```bash $ git commit ``` to commit precisely the changes you selected (the staged changes). Make sure you omit the `-a` option, otherwise Git will commit all the edits. What if you’ve edited many files in many places? Reviewing each change one by one becomes frustratingly mind-numbing. In this case, use `git add -i`, whose interface is less straightforward, but more flexible. With a few keystrokes, you can stage or unstage several files at a time, or review and select changes in particular files only. Alternatively, run `git commit --interactive` which automatically commits after you’re done. ``` #################### File: Git%20Magic%20-%20Ben%20Lynn%2C%20et%20al.%20%28HTML%2C%20PDF%2C%20EPUB%29.pdf Page: 32 Context: # The Index: Git’s Staging Area So far we have avoided Git's famous `index`, but we must now confront it to explain the above. The index is a temporary staging area. Git seldom shuttles data directly between your project and its history. Rather, Git first writes data to the index, and then copies the data in the index to its final destination. For example, `commit -a` is really a two-step process. The first step places a snapshot of the current state of every tracked file into the index. The second step permanently records the snapshot now in the index. Committing without the `-a` option only performs the second step, and only makes sense after running commands that somehow change the index, such as `git add`. Usually, we can ignore the index and pretend we are reading straight from and writing straight to the history. On this occasion, we want finer control, so we manipulate the index. We place a snapshot of some, but not all, of our changes into the index, and then permanently record this carefully rigged snapshot. ## Don’t Lose Your HEAD The `HEAD` tag is like a cursor that normally points at the latest commit, advancing with each new commit. Some Git commands let you move it. For example: ```bash $ git reset HEAD~3 ``` will move the `HEAD` three commits back. Thus all Git commands now act as if you hadn't made those last three commits, while your files remain in the present. See the help page for some applications. But how can you go back to the future? The past commits know nothing of the future. If you have the SHA1 of the original HEAD then: ```bash $ git reset 1b6a ``` But suppose you never took it down? Don’t worry: for commands like these, Git saves the original `HEAD` as a tag called `ORIG_HEAD`, and you can return safe and sound with: ```bash $ git reset ORIG_HEAD ``` ## HEAD-hunting Perhaps `ORIG_HEAD` isn’t enough. Perhaps you’ve just realized you made a monumental mistake and you need to go back to an ancient commit in a long-forgotten branch. By default, Git keeps a commit for at least two weeks, even if you ordered Git to destroy the branch containing it. The trouble is finding the appropriate hash. #################### File: Git%20Magic%20-%20Ben%20Lynn%2C%20et%20al.%20%28HTML%2C%20PDF%2C%20EPUB%29.pdf Page: 33 Context: You could look at all the hash values in `.git/objects` and use trial and error to find the one you want. But there's a much easier way. Git records every hash of a commit it computes in `.git/logs`. The subdirectory `refs` contains the history of all activity on all branches, while the file `HEAD` shows every hash value it has ever taken. The latter can be used to find hashes of commits on branches that have been accidentally lopped off. The `git reflog` command provides a friendly interface to these log files. Try: ```bash $ git reflog ``` Instead of cutting and pasting hashes from the reflog, try: ```bash $ git checkout "@{10 minutes ago}" ``` Or checkout the 5th-last visited commit via: ```bash $ git checkout "@{5}" ``` See the “Specifying Revisions” section of `git help rev-parse` for more. You may wish to configure a longer grace period for doomed commits. For example: ```bash $ git config gc.pruneexpire "30 days" ``` means a deleted commit will only be permanently lost once 30 days have passed and `git gc` is run. You may also wish to disable automatic invocations of `git gc`: ```bash $ git config gc.auto 0 ``` in which case commits will only be deleted when you run `git gc` manually. ## Building On Git In true UNIX fashion, Git's design allows it to be easily used as a low-level component of other programs, such as GUI and web interfaces, alternative command-line interfaces, patch management tools, importing and conversion tools, and so on. In fact, some Git commands are themselves scripts standing on the shoulders of giants. With a little tinkering, you can customize Git to suit your preferences. One easy trick is to use built-in Git aliases to shorten your most frequently used commands: ```bash $ git config --global alias.co checkout $ git config --global --get-regexp alias # display current aliases ``` ```bash alias.co checkout ``` ```bash $ git co foo # same as 'git checkout foo' ``` Another is to print the current branch in the prompt or window title. Invoking #################### File: Git%20Magic%20-%20Ben%20Lynn%2C%20et%20al.%20%28HTML%2C%20PDF%2C%20EPUB%29.pdf Page: 34 Context: ``` $ git symbolic-ref HEAD shows the current branch name. In practice, you most likely want to remove the `refs/heads/` and ignore errors: ```bash $ git symbolic-ref HEAD 2> /dev/null | cut -b 12- ``` The `contrib` subdirectory is a treasure trove of tools built on Git. In time, some of them may be promoted to official commands. On Debian and Ubuntu, this directory lives at `/usr/share/doc/git-core/contrib`. One popular resident is `workdir/git-new-workdir`. Via clever symlinking, this script creates a new working directory whose history is shared with the original repository: ```bash $ git-new-workdir an/existing/repo new/directory ``` The new directory and the files within can be thought of as a clone, except since the history is shared, the two trees automatically stay in sync. There's no need to merge, push, or pull. ## Daring Stunts These days, Git makes it difficult for the user to accidentally destroy data. But if you know what you are doing, you can override safeguards for common commands. ### Checkout Uncommitted changes cause checkout to fail. To destroy your changes, and checkout a given commit anyway, use the force flag: ```bash $ git checkout -f HEAD ``` On the other hand, if you specify particular paths for checkout, then there are no safety checks. The supplied paths are quietly overwritten. Take care if you use checkout in this manner. ### Reset Reset also fails in the presence of uncommitted changes. To force it through, run: ```bash $ git reset --hard 16d ``` ### Branch Deleting branches fails if this causes changes to be lost. To force a deletion, type: ```bash $ git branch -D dead_branch # instead of -d ``` Similarly, attempting to overwrite a branch via a move fails if data loss would ensue. To force a branch move, type: ```bash $ git branch -M source target # instead of -m ``` Unlike checkout and reset, these two commands defer data destruction. The changes are still stored in the `.git` subdirectory, and can be retrieved by `recover`. ``` #################### File: Git%20Magic%20-%20Ben%20Lynn%2C%20et%20al.%20%28HTML%2C%20PDF%2C%20EPUB%29.pdf Page: 35 Context: # Preventing Bad Commits Stupid mistakes pollute my repositories. Most frightening are missing files due to a forgotten `git add`. Lesser transgressions are trailing whitespace and unresolved merge conflicts; though harmless, I wish these never appeared on the public record. If only I had bought idiot insurance by using a hook to alert me about these problems: ```bash $ cd .git/hooks $ cp pre-commit.sample pre-commit # Older Git versions: chmod +x pre-commit ``` Now Git shorts a commit if useless whitespace or unresolved merge conflicts are detected. For this guide, I eventually added the following to the beginning of the `pre-commit` hook to guard against absent-mindedness: ```bash if git ls-files -o | grep '\.txt$'; then echo "FAIL! Untracked .txt files." exit 1 fi ``` Several git operations support hooks; see `git help hooks`. We activated the sample `post-update` hook earlier when discussing Git over HTTP. This runs whenever the head moves. The sample `post-update` script updates files Git needs for communication over Git-agnostic transports such as HTTP. # Secrets Revealed We take a peek under the hood and explain how Git performs its miracles. I will skimp over details. For in-depth descriptions refer to the user manual. ## Invisibility How can Git be so unobtrusive? Aside from occasional commits and merges, you can work as if you were unaware that version control exists. That is, until #################### File: Git%20Magic%20-%20Ben%20Lynn%2C%20et%20al.%20%28HTML%2C%20PDF%2C%20EPUB%29.pdf Page: 36 Context: # Integrity Most people associate cryptography with keeping information secret, but another equally important goal is keeping information safe. Proper use of cryptographic hash functions can prevent accidental or malicious data corruption. A SHA1 hash can be thought of as a unique 160-bit ID number for every string of bytes you'll encounter in your life. Actually more than that: every string of bytes that any human will ever use over many lifetimes. As a SHA1 hash is itself a string of bytes, we can hash strings of bytes containing other hashes. This simple observation is surprisingly useful: look up [hash chains](#). We'll later see how Git uses it to efficiently guarantee data integrity. Briefly, Git keeps your data in the `.git/objects` subdirectory, where instead of formal filenames, you'll find only IDs. By using IDs as filenames, as well as a few lockfiles and timestamping tricks, Git transforms any humble filesystem into an efficient and robust database. --- # Intelligence How does Git know you renamed a file, even though you never mentioned the fact explicitly? Sure, you may have run `git mv`, but that is exactly the same as a `git rm` followed by a `git add`. Git heuristically ferrets out renames and copies between successive versions. In fact, it can detect chunks of code being moved or copied around between files! Though it cannot cover all cases, it does a decent job, and this feature is always improving. If it fails to work for you, try options enabling more expensive copy detection, and consider upgrading. #################### File: Git%20Magic%20-%20Ben%20Lynn%2C%20et%20al.%20%28HTML%2C%20PDF%2C%20EPUB%29.pdf Page: 37 Context: # Indexing For every tracked file, Git records information such as its size, creation time, and last modification time in a file known as the index. To determine whether a file has changed, Git compares its current stats with those cached in the index. If they match, then Git can skip reading the file again. Since stat calls are considerably faster than file reads, if you only edit a few files, Git can update its state in almost no time. We stated earlier that the index is a staging area. Why is a bunch of file stats a staging area? Because the `add` command puts files into Git's database and updates these stats, while the `commit` command, without options, creates a commit based only on these stats and the files already in the database. ## Git's Origins This Linux Kernel Mailing List post describes the chain of events that led to Git. The entire thread is a fascinating archaeological site for Git historians. ## The Object Database Every version of your data is kept in the object database, which lives in the subdirectory `.git/objects`; the other residents of `.git` hold lesser data: the index, branch names, tags, configuration options, logs, the current location of the head commit, and so on. The object database is elementary yet elegant, and the source of Git's power. Each file within `.git/objects` is an object. There are 3 kinds of objects that concern us: **blob** objects, **tree** objects, and **commit** objects. ## Blobs First, a magic trick. Pick a filename, any filename. In an empty directory: ```bash $ echo sweet > YOUR_FILENAME $ git init $ git add $ find .git/objects -type f ``` You'll see `.git/objects/a/82372e8e745922cc69b36875a482cd3fd5c8d`. How do I know this without knowing the filename? It’s because the SHA1 hash of: ``` "blob" SP "6" NUL "sweet" LF ``` is `a82372e8e745922cc69b36875a482cd3fd5c8d`, where SP is a space, NUL is a zero byte, and LF is a linefeed. You can verify this by typing: ```bash $ printf "blob 6\000sweet\n" | shasum ``` #################### File: Git%20Magic%20-%20Ben%20Lynn%2C%20et%20al.%20%28HTML%2C%20PDF%2C%20EPUB%29.pdf Page: 38 Context: Git is content-addressable: files are not stored according to their filename, but rather by the hash of the data they contain, in a file we call a **blob object**. We can think of the hash as a unique ID for a file's contents, so in a sense we are addressing files by their content. The initial blob is merely a header consisting of the object type and its length in bytes; it simplifies internal bookkeeping. Thus I could easily predict what you would see. The file's name is irrelevant: only the data inside is used to construct the blob object. You may be wondering what happens to identical files. Try adding copies of your file, with any filenames whatsoever. The contents of `.git/objects` stay the same no matter how many you add. Git only stores the data once. By the way, the files within `.git/objects` are compressed with zlib, so you should not stare at them directly. Filter them through `zcat -d`, or type: ```bash git cat-file -p aa823728ea745922cc69368754a82d2df3d5c8d ``` which pretty-prints the given object. ## Trees But where are the filenames? They must be stored somewhere at some stage. Git gets around to the filenames during a commit: ```bash git commit -m "Type some message." find .git/objects -type f ``` You should now see 3 objects. This time I cannot tell you what the 2 new files are, as it partly depends on the filename you picked. We'll proceed assuming you chose *rose*. If you didn’t, you can rewrite history to make it look like you did: ```bash git filter-branch --tree-filter 'mv YOUR_FILENAME rose' find .git/objects -type f ``` Now you should see the file `.git/objects/05/b217b859794d08b9e4f704cbada4207be9` because this is the SHA1 hash of its contents: ``` tree "SP" "32" NULL "100644" NULL 0xa823728ea745922cc69368754a82d2df3d5c8d ``` Check this file does indeed contain the above by typing: ```bash echo 05b217b859794d08b9e4f704cbada4207be9 | git cat-file --batch ``` With zpipe, it's easy to verify the hash: ```bash zpipe -d .git/objects/05/b217b859794d08b9e4f704cbada4207be9 | shasum ``` Hash verification is trickier via `cat-file` because its output contains more than the raw uncompressed object file. This file is a **tree object**: a list of tuples consisting of a file type, a filename, and a hash. In our example, the file type is `100644`, which means *rose* is a normal file. #################### File: Git%20Magic%20-%20Ben%20Lynn%2C%20et%20al.%20%28HTML%2C%20PDF%2C%20EPUB%29.pdf Page: 41 Context: # SHA1 Weaknesses As time passes, cryptographers discover more and more SHA1 weaknesses. Already, finding hash collisions is feasible for well-funded organizations. Within years, perhaps even a typical PC will have enough computing power to silently corrupt a Git repository. Hopefully, Git will migrate to a better hash function before further research destroys SHA1. ## Microsoft Windows Git on Microsoft Windows can be cumbersome: - **Cygwin**, a Linux-like environment for Windows, contains a Windows port of Git. - **Git for Windows** is an alternative requiring minimal runtime support, though a few of the commands need some work. ## Unrelated Files If your project is very large and contains many unrelated files that are constantly being changed, Git may be disadvantaged more than other systems because single files are not tracked. Git tracks changes to the whole project, which is usually beneficial. A solution is to break up your project into pieces, each consisting of related files. Use `git submodule` if you still want to keep everything in a single repository. ## Who’s Editing What? Some version control systems force you to explicitly mark a file in some way before editing. While this is especially annoying when this involves talking to a central server, it does have two benefits: 1. Diffs are quick because only the marked files need be examined. 2. One can discover who else is working on the file by asking the central server who has marked it for editing. With appropriate scripting, you can achieve the same with Git. This requires cooperation from the programmer, who should execute particular scripts when editing a file. ## File History Since Git records project-wide changes, reconstructing the history of a single file requires more work than in version control systems that track individual files. #################### File: Git%20Magic%20-%20Ben%20Lynn%2C%20et%20al.%20%28HTML%2C%20PDF%2C%20EPUB%29.pdf Page: 43 Context: # Global Counter Some centralized version control systems maintain a positive integer that increases when a new commit is accepted. Git refers to changes by their hash, which is better in many circumstances. But some people like having this integer around. Luckily, it’s easy to write scripts so that with every update, the central Git repository increments an integer, perhaps in a tag, and associates it with the hash of the latest commit. Every clone could maintain such a counter, but this would probably be useless, since only the central repository and its counter matters to everyone. ## Empty Subdirectories Empty subdirectories cannot be tracked. Create dummy files to work around this problem. The current implementation of Git, rather than its design, is to blame for this drawback. With luck, once Git gains more traction, more users will clamour for this feature and it will be implemented. ## Initial Commit A stereotypical computer scientist counts from 0, rather than 1. Unfortunately, with respect to commits, Git does not adhere to this convention. Many commands are unfriendly before the initial commit. Additionally, some corner cases must be handled specially, such as rebasing a branch with a different initial commit. Git would benefit from defining the zero commit: as soon as a repository is constructed, HEAD would be set to the string consisting of 20 zero bytes. This special commit represents an empty tree, with no parent, at some time predating all Git repositories. Then running `git log`, for example, would inform the user that no commits have been made yet, instead of exiting with a fatal error. Similarly for other tools. Every initial commit is implicitly a descendant of this zero commit. However, there are some problem cases unfortunately. If several branches with different initial commits are merged together, then rebasing the result requires substantial manual intervention. ## Interface Quirks For commits A and B, the meaning of the expressions `"A..B"` and `"A...B"` depends on whether the command expects two endpoints or a range. See `git help diff` and `git help rev-parse`. #################### File: Git%20Magic%20-%20Ben%20Lynn%2C%20et%20al.%20%28HTML%2C%20PDF%2C%20EPUB%29.pdf Page: 44 Context: # Translating This Guide I recommend the following steps for translating this guide, so my scripts can quickly produce HTML and PDF versions, and all translations can live in the same repository. 1. **Clone the source**, then create a directory corresponding to the target language's IETF tag; see the W3C article on internationalization. For example, English is `en` and Japanese is `ja`. In the new directory, translate the text files from the `en` subdirectory. For instance, to translate the guide into Klingon, you might type: ```bash $ git clone git://repo.or.cz/gitmagic.git $ cd gitmagic $ mkdir tlh # "tlh" is the IETF language code for Klingon. $ cd tlh $ cp ../en/intro.txt . $ edit intro.txt # Translate the file. ``` and so on for each text file. 2. **Edit the Makefile** and add the language code to the `TRANSLATIONS` variable. You can now review your work incrementally: ```bash $ make tlh $ firefox book-tlh/index.html ``` 3. **Commit your changes often**, then let me know when they’re ready. GitHub has an interface that facilitates this: fork the "gitmagic" project, push your changes, then ask me to merge. #################### File: Project%20Oberon%20The%20Design%20of%20an%20Operating%20System%2C%20a%20Compiler%2C%20and%20a%20Computer%20-%20Niklaus%20Wirth%2C%20J%C3%BCrg%20Gutknecht%20%28PDF%29%20PO.Applications.pdf Page: 1 Context: # 10. The Network ## 10.1 Introduction Workstations are typically, but not always, connected in a local environment by a network. There exist two basically different views of the architecture of such nets. The more demanding view is that all connected stations constitute a single, unified workspace (also called address-space), in which the individual processors operate. It implies the demand that the "thin" connections between processors are hidden from the users. At worst they might become apparent through slower data access rates between the machines. To hide the difference between access within a computer and access between computers is regarded primarily as a challenge to implementors. The second, more conservative view, assumes that individual workstations are, although connected, essentially autonomous units which exchange data infrequently. Therefore, access of data on partner stations is initiated by explicit transfer commands. Commands handling external access are not part of the basic system, but rather are implemented in modules that might be regarded as applications. In the Oberon System, we adhere to this second view, and in this chapter, we describe the module **Net**, which is an autonomous command module based on the network driver SCC. It can be activated on any station connected in a network, and all of them are treated as equals. Such a set of loosely coupled stations may well operate in networks with moderate transmission rates and therefore with low-cost hardware interfaces and twisted-pair wires. An obvious choice for the unit of transferred data is the file. The central theme of this chapter is therefore file transfer over a network. Some additional facilities offered by a dedicated server station will be the subject of Chapter 11. The commands to be presented here are a few only: `SendFiles`, `ReceiveFiles`, and `SendMsg`. As explained in Chapter 2, Oberon is a single-process system where every command monopolizes the processor until termination. When a command involves communication over a network, (at least) two processors are engaged in the action at the same time. The Oberon paradigm therefore appears to exclude such cooperation; but fortunately it does not, and the solution to the problem is quite simple. Every command is initiated by a user operating on a workstation. For the moment we call it the **master** (of the command under consideration). The addressed station - obviously called the **server** - must be in a state where it recognizes the command in order to engage in its execution. Since the command - called a **request** - arrives in encoded form over the network, an Oberon task represented by a handler procedure must be inserted into the event polling loop of the system. Such a handler must have the general form: ``` IF event present THEN handle event END ``` The guard, in this case, must imply that a request was received from the network. We emphasize that the event is sensed by the server only after the command currently under execution, if any, has terminated. However, data arrive at the receiver immediately after they are sent by the master. Hence, any sizeable delay is inherently inadmissible, and the Oberon metaphor once again appears to fail. It does not fail, however, because the unavoidable, genuine concurrency of sender and receiver action is handled within the driver module which places the data into a buffer. The driver is activated by an interrupt, and its receiver buffer effectively decouples the partners and removes the stringent timing constraints. All this remains completely hidden within the driver module. ## 10.2 The Protocol If more than a single agent participates in the execution of a command, a convention must be established and obeyed. It defines the set of requests, their encoding, and the sequence of data. #################### File: Project%20Oberon%20The%20Design%20of%20an%20Operating%20System%2C%20a%20Compiler%2C%20and%20a%20Computer%20-%20Niklaus%20Wirth%2C%20J%C3%BCrg%20Gutknecht%20%28PDF%29%20PO.Applications.pdf Page: 3 Context: The source address is inserted automatically into packet headers by the driver. It is obtained from a DIP switch set when a computer is installed and connected. But where should the destination address come from? From the start, we reject the solution of an address table in every workstation because of the potential inconsistencies. The concept of a centralized authority holding a name/address dictionary is equally unattractive, because of the updates required whenever a person uses a different computer. Also, we have started from the premise to keep all participants in the network equal. The most attractive solution lies in a decentralized name service. It is based on the broadcast facility, i.e., the possibility to send a packet to all connected stations, bypassing their address filters with a special destination address (-1). The broadcast is used for emitting a name request containing the desired partner's symbolic name. A station receiving the request returns a reply to the requester, if that name matches its own symbolic name. The requester then obtains the desired partner's address from the source address field of the received reply. The corresponding simple protocol is: ``` NameRequest = NRQ partnername [NRS]. ``` Here, the already mentioned timeout facility is indispensable. The following summarizes the protocol developed so far: ``` protocol = {request}. request = ReceiveFile | SendFile | SendMsg | NameRequest. ``` The overhead incurred by name requests may be reduced by using a local address dictionary. In practice, a single entry is satisfactory. A name request is then needed whenever the partner changes. ## 10.4 The Implementation Module `Net` is an implementation of the facilities outlined above. The program starts with a number of auxiliary, local procedures. They are followed by procedure `Serve` which is to be installed as an Oberon task, and the commands `SendFile`, `ReceiveFiles`, and `SendMsg`, each of which has its counterpart within procedure `Serve`. At the end are the commands for starting and stopping the server task. For a more detailed presentation we select procedure `ReceiveFiles`. It starts out by reading the first parameter which designates the partner station from the command line. Procedure `FindPartner` issues the name request, unless the partner's address has already been determined by a previous command. The global variable `partner` records a symbolic name (ID) whose address is stored in the destination field of the global variable `head`, which is used as header in every packet sent by procedure `SCC.SendPacket`. The variable `partner` may be regarded as a name cache with a single entry and with the purpose of reducing the number of issued name requests. If the partner has been identified, the next parameter is read from the command line. It is the name of the file to be transmitted. If the parameter has the form `name@name`, the file stored on the server as `name.name` is fetched and stored locally as `name`. Hence, `name` serves as a prefix of the file name on the server station. Thereafter, the requested parameters are concatenated in the local buffer variable `buf`. They are the user’s name and password followed by the file name (user name and password remain unused by the server procedures defined here). The command package is dispatched by the call `Send(SND, buf)`, where `buf` denotes the length of the command parameter string. Then, the reply packet is awaited by calling `ReceiveHead`. If the received packet's type is `DAT` with sequence number 0, a new file is established. Procedure `ReadData` receives the data and stores them in the new file, obeying the protocol defined in Section 10.2. This process is repeated for each file specified in the list of file names in the command line. ### Procedure `ReceiveHead()` Receives packets and discards them until one arrives from the partner from which it is expected. The procedure represents an input filter in addition to the one provided by the protocol. #################### File: Project%20Oberon%20The%20Design%20of%20an%20Operating%20System%2C%20a%20Compiler%2C%20and%20a%20Computer%20-%20Niklaus%20Wirth%2C%20J%C3%BCrg%20Gutknecht%20%28PDF%29%20PO.Applications.pdf Page: 4 Context: # Data Transmission Procedures The hardware discriminates on the basis of the packets' source address, whereas the hardware filter discriminates on the basis of the destination address. If no packet arrives within the allotted time `T`, a type code `-1` is returned, signifying a timeout. ## Procedure ReceiveData The `ReceiveData` checks the sequence numbers of incoming data packets (type 0 - 7). If an incorrect number is detected, an ACK packet with the previous sequence number is returned (type 16 - 23), requesting a retransmission. At most two retries are undertaken. This seems to suffice, considering that also the server does not accept any other requests while being engaged in the transmission of a file. The part corresponding to `ReceiveFiles` within procedure `Serve` is guarded by the condition `head.tpe = SND`. Variable `head` is the recipient of headers whenever a packet is received by `ReceiveHead`. First, the request's parameters are scanned. `Id` and `pw` are ignored. Then the requested file is opened. If it exists, the transmission is handled by `ReceiveData`'s counterpart, procedure `SendData`. The time limit for receiving the next data packet is `T1`, whereas the limit of number of possible (re)transmissions is `T0`. `T1` is roughly `70` multiplied by the maximum data wait until no further retransmission requests can be expected to arrive. The value `T0 (300)` corresponds to `1`; the time for transmission of a packet of maximum length is about `16ms`. ## Procedure SendFiles Procedure `SendFiles` is designed analogously; its counterpart in the server is guarded by the condition `head.tpe = REC`. The server accepts the request only if its state is unprotected (global variable `protected`). Otherwise, the request is negatively acknowledged with an NPR packet. We draw attention to the fact that procedures `SendData` and `ReceiveData` are both used by command procedures as well as by the server. #################### File: Project%20Oberon%20The%20Design%20of%20an%20Operating%20System%2C%20a%20Compiler%2C%20and%20a%20Computer%20-%20Niklaus%20Wirth%2C%20J%C3%BCrg%20Gutknecht%20%28PDF%29%20PO.Applications.pdf Page: 5 Context: # 11. A Dedicated File-Distribution and Mail-Server ## 11.1. Concept and Structure In a system of loosely coupled workstations, it is desirable to centralize certain services. A first example is a common file store. Even if every station is equipped with a disk for permanent data storage, a common file service is beneficial, e.g., for storing the most recent versions of system files, reference documents, reports, etc. A common repository avoids inconsistencies which are inevitable when local copies are created. We call this a **file distribution service**. A centralized service is also desirable if it requires equipment whose cost and service would not warrant its acquisition for every workstation, particularly if the service is infrequently used. A prime example of this case is a **printing service**. The third case is a communication facility in the form of **electronic mail**. The repository of messages must inherently be centralized. We imagine it to have the form of a set of mailboxes, one for each user in the system. A mailbox needs to be accessible at all times, i.e., also when its owner’s workstation has been switched off. A last example of a centralized service is a **time server**. It allows a station's real time clock to be synchronized with a central clock. In passing, we point out that every user has full control over his station, including the right to switch it on and off at any time. In contrast, the central server is continuously operational. In this chapter, we present a set of server modules providing all the above-mentioned services. They rest on the basic Oberon System without module Net (see Chapter 10). In contrast to **Net**, module **NetServer**, which handles all network communication, contains no command procedures (apart from those for starting and stopping it). This is because it never acts as a master. The counterparts of its server routines reside in other modules, including an extended version of **Net**, on the individual workstations. Routines for the file distribution service are the same as those contained in module **Net**, with the addition of permission checks based on the received user names and passwords. Routines for printing and mail service could in principle also be included in **NetServer** in the same way. But considerations of reliability and timing made this simple solution appear unsatisfactory. A weaker coupling in line of data transmission and data consumption is indeed highly desirable. Therefore, files received for printing or for dispatching into mailboxes are stored (by **NetServer**) in temporary files and thereafter "handed over" to the appropriate agent, i.e., the print server or the mail server. This data-centered interface between servers—in contrast to procedural interfaces—has the advantage that the individual servers are independent in the sense that none imports any other. Therefore, their development could proceed autonomously. Their connection is instead a module that defines a data structure and associated operators for passing temporary files from one server to another. The data structure used for this purpose is the first-in-first-out queue. We call its elements tasks, because each one carries an objective and an object, the file to be processed. The module containing the FIFOs is called **Core**. The resulting structure of the involved modules is shown in Figure 11.1. ![Figure 11.1](link_to_figure) Figure 11.1 includes another server, **LineServer**, and shows the ease with which additional servers may be inserted in this scheme. They act as further sources and/or sinks for tasks, feeding or consuming the queues contained in **Core**. **LineServer** indeed produces and consumes tasks like **NetServer**. Instead of the RS-485 bus, it handles the RS-232 line which, connected to a modem, allows access to the server over telephone lines. We refrain from describing this module in further detail, because in many ways it is a mirror of **NetServer**. A centralized, open server calls for certain protection measures against unauthorized use. We recall that requests always carry a user identification and a password as parameters. The server #################### File: Project%20Oberon%20The%20Design%20of%20an%20Operating%20System%2C%20a%20Compiler%2C%20and%20a%20Computer%20-%20Niklaus%20Wirth%2C%20J%C3%BCrg%20Gutknecht%20%28PDF%29%20PO.Applications.pdf Page: 6 Context: ``` checks their validity by examining a table of users. The respective routines and the table are contained in module Core (see Sect. 11.5). ``` **Figure 11.1** Module structure of server systems | | | | | | |---------|---------------|-------------|-----------|----------| | NetServer | LineServer | PrintServer | MailServer | Core | | SCC | RS232 | PrintMaps | | | ## 11.2. Electronic Mail Service The heart of an e-mail service is the set of mailboxes stored on the dedicated, central server. Each registered user owns a mailbox. The evidently necessary operations are the insertion of a message and its retrieval. In contrast to customary letter boxes, however, a retrieved message need not necessarily be removed from the box; its retrieval produces a copy. The box thereby automatically becomes a repository, and messages can be retrieved many times. This scheme calls for an additional command which removes a message from the box. Also, a command is needed for delivering a table of contents, in which presumably each message is represented by an indication of its send time and time of arrival. The mail scheme suggested above results in the following commands: - `Net.Mailbox ServerName`. This command fetches a table of contents of the current user's mailbox from the specified server and displays it in a new viewer. The user's name and password must have been registered previously by the command `System.SetUser`. - `Net.SendMail ServerName`. The text in the marked viewer is sent to the specified server. In order to be accepted, the text must begin with at least one line beginning with "To:" and containing at least one recipient. - `Net.ReceiveMail`. This command is contained in the title bar (menu) of the viewer abandoned when requesting the table of contents. Prior to issuing the command, the message to be read must have been specified by selecting a line in the table of contents in this viewer. This mail system presented here is primarily intended to serve as an exchange for short messages which are typically sent, received, read, and discarded. Mailboxes are not intended to serve as long term archives for a large and ever-growing number of long pieces of text. This restrictiveness of purpose allows to choose a reasonably simple implementation and results in efficient, practically instantaneous access to messages when the server is idle. The Oberon mail server used at ETH also provides communication with external correspondents. It connects to an external mail server which is treated as a source and a sink for messages (almost) like other customers. Additionally, messages sent to that server need to be encoded into a standardized format, and those received need to be decoded accordingly. The parts of module `MailServer` for encoding and decoding are not described in this book. We merely outline the design and implementation took a multiple of the time spent on the last, local message exchange, to which we confine this presentation. From the structures explained in Section 11.1, it follows that three agents are involved in the transfer of messages from the user into a mailbox. Therefore, additions to the server system distribute over three modules. New commands are added to module `Net` (see Section 10.4); these procedures will be listed below. Their counterparts reside in module `NetServer` on the dedicated server. ``` #################### File: Project%20Oberon%20The%20Design%20of%20an%20Operating%20System%2C%20a%20Compiler%2C%20and%20a%20Computer%20-%20Niklaus%20Wirth%2C%20J%C3%BCrg%20Gutknecht%20%28PDF%29%20PO.Applications.pdf Page: 8 Context: # Queue ``` last | v +---+ | n | | 3 | +---+ | v +---+ | | +---+---+---+ | | v v 8 15 | id | jg | hm | rw +---+---+---+ | v NIL ``` **Figure 11.3 Structure of task queue** Every mailbox is represented as a file. This solution has the tremendous advantage that no special administration has to be introduced to handle a reserved partition of disk store for mail purposes. A mailbox file is partitioned into three parts: the block reservation part, the directory part, and the message part. Each part is quickly locatable, because the first two have a fixed length (32 and 31*32 = 992 bytes). The message part is regarded as a sequence of blocks (of 256 bytes), and each message occupies an integral number of adjacent blocks. Corresponding to each block, the block reservation part contains a single bit indicating whether or not the block is occupied by a message. Since the block reservation part is 32 bytes long, the message part contains at most 256 blocks, i.e. 64K bytes. The block length was chosen after an analysis of messages which revealed that the average message is less than 500 bytes long. The directory part consists of an array of 31 elements of type `MailEntry`, a record with the following fields: - `pos` and `len` indicate the index of the message's first block and the message's number of bytes; - `time` and `date` indicate the message's time of insertion, and `originator` indicates the message's source. The entries are linked (field `next`) in chronological order of their arrival, and entry 0 serves as the list's head. It follows that a mailbox contains at most 30 messages. An example of a mailbox table is shown in Figure 11.4. ## MailEntry ``` MAILEntry = RECORD pos: next: INTEGER; len: LONGINT; time: INTEGER; date: ARRAY 20 OF CHAR; END; ``` ## MResTab ``` MResTab = ARRAY 31 OF SET; MailDir = ARRAY 31 OF MailEntry; ``` We are now in a position to implement the handler for requests for message retrieval. It is guarded by the condition `top := SML`. After a valid check, the respective requestor's mailbox file is opened. The mailbox opened is related by the global variable `MF` which acts as a single entry cache. The associated user number is given by the global variable `mailno`. Since typically several requests involving the same mailbox follow, this measure avoids the repeated reopening of the same file. Thereafter, a rider is directly positioned at the respective directory entry for reading the message's length and position in the message part. The rider is repositioned accordingly, and transmission of the message is handled by procedure `SendMail`. #################### File: Project%20Oberon%20The%20Design%20of%20an%20Operating%20System%2C%20a%20Compiler%2C%20and%20a%20Computer%20-%20Niklaus%20Wirth%2C%20J%C3%BCrg%20Gutknecht%20%28PDF%29%20PO.Applications.pdf Page: 9 Context: # Block reservation part ``` 1 100000010111110111111 ``` # Directory part | pos | len | time | date | orig | next | |-----|-----|---------|---------|-------------|--------| | 0 | 0 | 8.92 | 10:12 | 15.21 | Muller | | 1 | 2 | 15.97 | 11:27.2 | 17.19 | Templ | | 12 | 20 | 2.00 | 23:41 | 6.61 | Franz | # Message part ``` 0 2 8 15 ``` **Figure 11.4 State of mailbox file** Requests for the mailbox directory are handled by the routine guarded by the condition `typ = MDIR`. The directory part must be read and converted into a text. This task is supported by various auxiliary procedures (`Append`) which concatenate supplied data in a buffer for latter transmission. We emphasize that this request does not require the reading of any other part of the file, and therefore is very swift. The last of the four main service requests (DML) deletes a specified message. Removal from the directory requires a relinking of the entries. Unused entries are marked by their `len` field having a value of 0. Also, the blocks occupied by the message become free. The block reservation part must be updated accordingly. In passing, we note that the use of files for representing mailboxes, in combination with the file distribution services residing on the same server station, allows anyone to access (and inspect) any mailbox. Although we do not claim that this system provides secure protection against snooping, a minimal effort for protection was undertaken by a simple encoding of messages in mailbox files. This encoding is not shown in the program listings contained in this book. One operation remains to be explained in more detail: the processing of tasks inserted into the mail queue. It consists of the insertion of the message represented by the task's file into one or several mailboxes. It involves the interpretation of the message's header, i.e., lines containing addresses, and the construction of a new header containing the name of the originator and the date of insertion into the mailbox. These actions are performed by procedures in module `MailServer`. Its procedure `Serve` is installed as an Oberon Task, and it is guarded by the condition `Core.MailQueue.n > 0`, indicating that at least one message needs to be dispatched. The originator's name is obtained from `Core.GetUserName(uno)`, where `uno` is the user number obtained from the queue entry. The actual time is obtained from `Oberon.GetClock`. The form of the new header is shown by the following example: **From:** Gutknecht **Date:** 12.08.91 09:34:15 The received message's header is then searched for recipients. Their names are listed in header lines starting with "To:" (or "Cc:"). After a name has been read, the corresponding user number is obtained by calling `Core.UserNum`. Then the message is inserted into the designated mailbox by procedure `Dispatch`. The search for recipients continues, until a line is encountered that does not begin with "To:" (or "Cc:"). A negative user number indicates that the given name is not registered. #################### File: Project%20Oberon%20The%20Design%20of%20an%20Operating%20System%2C%20a%20Compiler%2C%20and%20a%20Computer%20-%20Niklaus%20Wirth%2C%20J%C3%BCrg%20Gutknecht%20%28PDF%29%20PO.Applications.pdf Page: 11 Context: # Summary of Protocol: ## Protocol | Protocol | Request | |-------------------|--------------------------------------------------------| | | ReceiveFile | SendFile | DeleteFile | Directory | | | MailBox | SendMail | ReceiveMail | DeleteMail | | | PrintStream | Sending | NameRequest | NewPassword | GetTime | ### ReceiveFile - SND username password filename (datastream | NAK | NPN). - datastream = DATA data ACK1 (DATA data ACK1:). ### SendFile - REC username password filename (ACK0 datastream | NAK | NRP). - datastream = DATA data ACK1 (DATA data ACK1:). ### DeleteFile - DEL username password filename (ACK | NAK | NRP). ### Directory - FDIR username password prefix (datastream | NAK | NRP). ### MailBox - FDIR username password (datastream | NAK | NRP). ### SendMail - RML username password (ACK datastream | NAK | NRP). ### ReceiveMail - SML username password msgno (datastream | NAK | NRP). ### DeleteMail - DML username password msgno (ACK datastream | NAK | NRP). ### PrintStream - PRT username password (ACK datastream | NAK | NRP). ### SendMsg - MSG message ACK. ### NameRequest - NRQ username (NRS). ### NewPassword - NWP username password (ACK DAT newpassword (ACK | NAK | NRP)). ### GetTime - TKN TIM time date. ## 11.5 User Administration It appears to be a universal fact that centralization inevitably calls for an administration. The centralized mail and printing services make no exception. The typical duties of an administration are accounting and protection against misuse. It has to ensure that registered services are counted and that an unauthorized user is taking advantage of the server. An additionally obvious item is that of statistical data. In our case, accounting plays a very minor role, and the reason for the existence of the administration presented here is primarily protection. We distinguish between two kinds of protection. The first is protection of the server's resources in general, the second is that of individual users' resources from being accessed by others. Whereas in the first case some validation of a user’s identification might suffice, the second case requires the association of personal resources with user names. In any case, the central server must store data for each member of the set of registered users. Specifically, it must be able to check the admissibility of a user's request on the basis of stored information. Evidently, a protection administration is similar in purpose and function to a lock. Quite regularly, locks are subjected to attempts of breaking them, and locks can also be subjected to attempts of being outwitted. The race between techniques of breaking locks and that of better countermeasures is well known, and we do not even try to make a contribution to it. Our design is based on the premise that the Oberon Server operates in a harmonious environment. Nevertheless, a minimal amount of protection machinery was included. It raises the amount of effort required for breaking protection to a level which is not reached when curiosity alone is the motivation. The data about users is held in a table in module Core. As mentioned earlier, Core acts as the connector between the various servers by means of task queues. Its designed purpose is to provide the necessary access to user data via appropriate procedures. In the simplest solution, each table entry would contain a user name only. For each request, the administration would merely test for the presence of the request's user name in the table. A significant step towards safe protection is the introduction of a password in addition to the user name. In order that a request be honored, not only must the name be registered, but the delivered and the stored password must match. Evidently, abusive attempts would aim at recovering the #################### File: Project%20Oberon%20The%20Design%20of%20an%20Operating%20System%2C%20a%20Compiler%2C%20and%20a%20Computer%20-%20Niklaus%20Wirth%2C%20J%C3%BCrg%20Gutknecht%20%28PDF%29%20PO.Applications.pdf Page: 12 Context: # User Identification and Password Management Stored passwords are managed through our solution that stores an encoded password. The command `System.SetUser`, which asks for a user identification and a password, immediately encodes the password. The original is stored nowhere; the encoding algorithm is such that it is difficult to construct a corresponding decoder. ## User Identification The mail service requires a third attribute in addition to identification and encoded password: the user's name as it is used for addressing messages. Identification typically consists of the user's initials; for the name, we suggest the full last name of the user and disambiguate cryptic abbreviations. ## Printing Service The printing service makes accounting facility desirable. A fourth field in each user table entry serves as a count for the number of printed pages. As a result, there are four fields: `id`, `name`, `password`, and `count`. The table is not exported but only accessible via procedures. `Core` is a good example of a resource hiding module. The program is listed below, and a few additional comments follow here. ### Procedures - `UserNo(id)` and `UserNum(name)` yield the table index of the identified user; it is called the user number and is used as a short encoding for recipients and senders within the mail server. In other servers, the number is merely used to check a request’s validity. ### User Information User information must certainly survive any interruption of server operation, be it due to software, hardware, or power failure. This requires that a copy of the user information is held on backup storage (disk). The simplest solution would be to use a file for this purpose. However, this would indeed make protection too vulnerable: files can be accessed easily, and we have refrained from introducing a file protection facility. Instead, the backup of the user information is held on a few permanently reserved sectors on the server machine, which are inaccessible to the file system. ## Administrative Procedures Apart from procedures and variables constituting the queuing mechanism for tasks, the procedures exported from module `Core` all belong to the administration, and they can be divided into two categories: 1. **User Management**: - `UserNo`, `UserNum`, `IncPageCount`, `SetPassword`, `GetUserName`, `GetFileName`. 2. **User Inspection**: - `ListUsers`, `GetUsers`, and `Init` for making changes to the table. The client of the latter category is a module `Users` which is needed by the human administrator of the server facility. ## Conclusion The reader may at this point wonder why a more advanced concept of administration has not been chosen, which would allow the human administrator to operate the server remotely. A quick analysis of the consequences of this widely used approach reveals that a substantial amount of additions to our system would be required. The issue of security and protection would become inflated into dimensions that are hardly justified for our local system. The first consequence would be a differentiation among levels of protection. The administrator would become a so-called super-user with extra privileges, such as changing the user table. And so the game of trying to break the protection measures starts to become an interesting challenge. We have resisted the temptation to introduce additional complexity. Instead, we assume that physical access to the server station is reserved to the administrator. Naturally, module `Users` and in particular the symbol file of `Core` do not belong to the public domain. In concluding, we may point out that the impossibility of activating users’ programs on the server station significantly reduces the possibilities for inflicting damage from the exterior. #################### File: Project%20Oberon%20The%20Design%20of%20an%20Operating%20System%2C%20a%20Compiler%2C%20and%20a%20Computer%20-%20Niklaus%20Wirth%2C%20J%C3%BCrg%20Gutknecht%20%28PDF%29%20PO.Applications.pdf Page: 13 Context: # 12 The Compiler ## 12.1 Introduction The compiler is the primary tool of the system builder. It therefore plays a prominent role in the Oberon System, although it is not part of the basic system. Instead, it constitutes a tool module - an application - with a single command: `Compile`. It translates program texts into machine code. Therefore, it is a program inherently machine-dependent; it acts as the interface between source language and target computer. In order to understand the process of compilation, the reader needs to be familiar with the source language Oberon defined in Appendix 1, and with the target computer RISC, defined in Appendix 2. The language is defined as an infinite set of sequences of symbols taken from the language's vocabulary. It is described by a set of equations called syntax. Each equation defines a syntactic construct, or more precisely, the set of sequences of symbols belonging to that construct. It specifies how that construct is composed of other syntactic constructs. The meaning of programs is defined in terms of semantic rules governing each such construct. Compilation of a program text proceeds by analyzing the text and thereby decomposing it recursively into its constructs according to the syntax. When a construct is identified, code is generated according to the semantic rule associated with the construct. The components of the identified construct supply parameters for the generated code. It follows that we distinguish between two kinds of actions: analyzing steps and code generating steps. In a rough approximation we may say that the former are source language dependent and target computer dependent, whereas the latter are source language independent and target computer dependent. Although reality is somewhat more complex, the module structure of this compiler clearly reflects this division. The main module of the compiler is ORP (for Oberon to RISC Parser). It is primarily dedicated to syntactic analysis, parsing. Upon recognition of a syntactic construct, an appropriate procedure is called to the code generator module ORG (for Oberon to RISC Generator). Apart from parsing, ORP checks for type consistency of operands, and it computes the attributes of objects identified in declarations. Whereas ORP mirrors the source language and is independent of a target computer, ORG reflects the target computer, but is independent of the source language. Oberon program texts are regarded as sequences of symbols rather than sequences of characters. Symbols themselves, however, are sequences of characters. We refrain from explaining the reasons for this distinction, but mention that apart from special characters and pairs such as `.`, `,`, as well as identifiers, numbers, and strings are classified as symbols. Furthermore, certain capital letter sequences are symbols, such as `IF`, `END`, etc. Each time the syntax analyzer (parser) proceeds to read the next symbol, it does this by calling procedure `Get`, which constitutes the so-called scanner residing in module ORS (for Oberon to RISC Scanner). It reads from the source text as many characters as are needed to recognize the next symbol. In passing, we note that the scanner also reflects the definition of symbols in terms of characters, whereas the parser is based on the notion of symbols only. The scanner implements the abstraction of symbols. The recognition of symbols within a character sequence is called **lexical analysis**. Ideally, the recognition of any syntactic construct, say `A`, consisting of subconstructs, say `B1`, `B2`, ..., `Bn`, leads to the generation of code that depends only on (1) the semantic rules associated with `A`, and (2) on attributes of `B1`, `B2`, ..., `Bn`. If this condition is satisfied, the construct is said to be context-free, and if all constructs of a language are context-free, then the language is context-free. Syntax and semantics of Oberon adhere to this rule, although with a significant exception. #################### File: Project%20Oberon%20The%20Design%20of%20an%20Operating%20System%2C%20a%20Compiler%2C%20and%20a%20Computer%20-%20Niklaus%20Wirth%2C%20J%C3%BCrg%20Gutknecht%20%28PDF%29%20PO.Applications.pdf Page: 14 Context: # Compiler / Parser ORP ## Code generator ORG ### Table handler ORB - Scanner ORS - Texts, Oberon - Files ![Figure 12.1 Compiler's module structure](path/to/image) ## 12.2 Code patterns Before it is possible to understand how code is generated, one needs to know **which** code is generated. In other words, we need to know the goal before we find the way leading to the goal. A fairly concise description of this goal is possible due to the structure of the language. As explained before, semantics are attached to each individual syntactic construct, independent of its context. Therefore, it suffices to list the expected code—instead of an abstract semantic rule—for each syntactic construct. As a prerequisite to understanding the resulting instructions and in particular their parameters, we need to know where declared variables are stored, i.e., which are their addresses. This compiler #################### File: Project%20Oberon%20The%20Design%20of%20an%20Operating%20System%2C%20a%20Compiler%2C%20and%20a%20Computer%20-%20Niklaus%20Wirth%2C%20J%C3%BCrg%20Gutknecht%20%28PDF%29%20PO.Applications.pdf Page: 15 Context: # Variable Allocation in RISC Processors In RISC processors, variable allocation uses a straightforward scheme of sequential allocation of consecutively declared variables. An address is a pair consisting of a base address (in a register) and an offset. Global variables are allocated in the module's data section and the respective base address register is SB (Static Base, see Chapter 6). Local variables are allocated in a procedure activation record on the stack; the respective base register is SP (Stack Pointer). Offsets are positive integers. The amount of storage needed for a variable (called its *size*) is determined by the variable's type. The sizes of basic types are prescribed by the target computer's data representation. The following holds for the RISC processor: | Type | No. of bytes | |--------------------------|--------------| | BYTE, CHAR, BOOLEAN | 1 | | INTEGER, REAL, SET, POINTER, PROCEDURE | 4 | The size of an array is the size of the element type multiplied by the number of elements. The size of a record is the sum of the sizes of its fields. A complication arises due to so-called alignment. By alignment is meant the adjustment of an address to a multiple of the variable's size. Alignment is performed for variable addresses as well as for record field offsets. The motivation for alignment is the avoidance of double memory references for variables being "distributed" over two adjacent words. Proper alignment enhances processing speed quite significantly. Variable allocation using alignment is shown by the example in Fig. 12.2. ## Example of Variable Allocation ``` VAR b0: BYTE; int0: INTEGER; b1: BYTE; int1: INTEGER; ``` ``` b0 int0 | b1 | int1 | -------------- int0 b1 int1 ``` ### Figure 12.2: Alignment of variables We note in passing that a reordering of the four variables lessens the number of unused bytes, as shown in Fig. 12.3. ``` VAR int0, int1: INTEGER; b0, b1: BYTE; ``` ``` int0 int1 b1 | b0 ``` ### Figure 12.3: Improved order of variables Memory instructions compute the address as the sum of a register (base) and an offset constant. Local variables use the *stack pointer* SP (R14) as base, global variables the static base SB (R13). Every module has its own SB value, and therefore access to global (and imported) variables requires two instructions, one for fetching the base value, and one for loading or storing data. If the compiler can determine whether the correct base value has already been loaded into the SB register, the former instruction is omitted. The first 7 sample patterns contain global variables only, and their base SB is assumed to hold the appropriate value. Parameters of branch instructions denote jump distances from the instruction's own location (PC-relative). #################### File: Project%20Oberon%20The%20Design%20of%20an%20Operating%20System%2C%20a%20Compiler%2C%20and%20a%20Computer%20-%20Niklaus%20Wirth%2C%20J%C3%BCrg%20Gutknecht%20%28PDF%29%20PO.Applications.pdf Page: 16 Context: ``` # Pattern 1: Assignment of constants We begin with a simple example of assigning constants to variables. The variables used in this example are global; their base register is SB. Each assignment results in a single instruction. The constant is embedded within the instruction as a literal operand. ## MODULE Pattern1: VAR ch: CHAR; 0 k: INTEGER; 4 x: REAL; 8 s: SET; 12 BEGIN // module entry code ch := '0'; // 40000030 MOV R0 30H k := 10; // 40000000 MOV R0 10 x := 1.0; // 600003F8 MOV R0 3F8000H s := [0, 4, 8]; // 40000111 MOV R0 111H END Pattern1. // module exit code # Pattern 2: Simple expressions The result of an expression containing operators is always stored in a register before it is assigned to a variable or used in another operation. Registers for intermediate results are allocated sequentially in ascending order R0, R1, ..., R11. Integer multiplication and division by powers of 2 are represented by shifts (LSL, ASR). Similarly, the modulus by a power of 2 is obtained by masking off leading bits. The operations of set union, difference, and intersection are represented by logical operations (OR, AND). ## MODULE Pattern2: VAR i, j, k: INTEGER; 0, 4, 8 x, y: REAL; 16, 20 s: SET; 24, 28, 32 BEGIN i := (i + 1) * (i - 1); // LDR R0 S0 // ADD R0 R0 1 // LDR R1 S0 // SUB R1 R1 1 // MUL R0 R0 R1 k := k DIV 17; // LDR R0 S8 // DIV R0 R0 17 // STR R0 S8 k := 8**n; // LDR R0 S12 // LSL R0 R0 3 // STR R0 S8 k := n DIV 2; // LDR R0 S12 // ASR R0 R0 1 // STR R0 S8 k := n MOD 16; // LDR R0 S12 // AND R0 S15 // STR R0 S8 x := -y / (x - 1.0); // LDR R0 S16 // MOV R0 R0 3F80H // FSB R0 R0 R1 // LDR R1 S20 // FDIV R0 R1 R0 s := s + t + u; // FSB R0 R1 0 // STR R0 S16 // LDR R0 S28 // LDR R1 S32 END Pattern2. // module exit code ``` #################### File: Project%20Oberon%20The%20Design%20of%20an%20Operating%20System%2C%20a%20Compiler%2C%20and%20a%20Computer%20-%20Niklaus%20Wirth%2C%20J%C3%BCrg%20Gutknecht%20%28PDF%29%20PO.Applications.pdf Page: 17 Context: # Pattern3: Indexed Variables References to elements of arrays make use of the possibility to add an index value to an offset. The index must be present in a register and multiplied by the size of the array elements. (For integers with size 4 this is done by a shift of 2 bits). Then the index is checked whether it lies within the bounds specified in the array's declaration. This is achieved by a comparison, actually a subtraction, and a subsequent branch instruction causing a trap, if the index is either negative or beyond the upper bound. If the reference is to an element of a multi-dimensional array (matrix), its address computation involves several multiplications and additions. The address of an element A[i1, ..., ik] of a k-dimensional array A with lengths n1, ..., nk is: ``` A(i) + ((...(((i1 * n2) + i2) * n3 + ...) * n_k) + i_k) * n1) + n0 ``` Note that for index checks `CMP` is written instead of `SUB` to mark that the subtraction is merely a comparison, that the result remains unused and only the condition flag registers hold the result. ## MODULE Pattern3: ```assembly VAR i, j, k: INTEGER; 0, 4, 8, 12 a: ARRAY 10 OF INTEGER; 16 x: ARRAY 10, 10 OF INTEGER; 56 y: ARRAY 10, 10 OF INTEGER; 456 BEGIN k := a[j]; LDR R0 S8 CMP R1 10 BLH R12 LSL R0 2 ADD R0 S8 R0 LDR R0 R0 16 STR R0 S8 n := a[5]; LDR R0 S8 STR R0 S8 12 x[i, j] := 2; LDR R0 S8 CMP R1 10 BLH R12 MUL R0 R0 40 ADD R0 R0 S8 CMP R2 R1 10 BLH R12 LDR R0 S8 4 y[i, j] := k - 3; LDR R0 S8 CMP R1 10 BLH R12 MUL R1 R0 400 ADD R0 R0 S8 LDR R1 S8 4 CMP R2 R1 10 END Pattern3. ``` #################### File: Project%20Oberon%20The%20Design%20of%20an%20Operating%20System%2C%20a%20Compiler%2C%20and%20a%20Computer%20-%20Niklaus%20Wirth%2C%20J%C3%BCrg%20Gutknecht%20%28PDF%29%20PO.Applications.pdf Page: 18 Context: # BLH R12 LSL R1 R12 ADO R0 R0 R1 MOV R1 R0 3 STR R1 R0 456 MOV R0 R6 STR R0 SB 1836 END Pattern 3. ## Pattern 4: Record fields and pointers Fields of records are accessed by computing the sum of the record's (base) address and the field's offset. If the record variable is statically declared, the sum is computed by the compiler. ```pascal MODULE Pattern4; TYPE Ptr = POINTER TO Node; Node = RECORD num: INTEGER; 0 name: ARRAY [0..4] OF CHAR; 4 next: Ptr; 12 END; VAR p: Ptr; 12, 16 q: Node; 20 BEGIN r.num := 10; MOV R0 R10 STR R0 SB 20 p.num := 6 LDR R0 SB 12 (p) MOV R1 R0 6 STR R1 R0 0 LDR R0 SB 12 MOV R1 R0 30 STR R1 R0 11 (4+7) p.next := q; LDR R0 SB 12 STR R1 R0 12 p.next.next := NIL LDR R0 SB 12 LDR R0 R0 12 (p.next) MOV R1 R0 0 (NIL) STR R1 R0 12 (p.next.next) END Pattern4. ``` ## Pattern 5: Boolean expressions, IF statements Conditional statements imply that parts of them are skipped. This is done by the use of branch instructions whose operand specifies the distance of the branch. The instructions refer to the condition-register as an implicit operand. Its value is determined by a preceding instruction, typically a compare or a bit-test instruction. The Boolean operators `&` and `OR` are purposely not defined as total functions, but rather by the equations: ``` p & q = if p then q else FALSE p OR q = if p then TRUE else q ``` Consequently, Boolean operators must be translated into branches too. Evidently, branches stemming from if statements and branches stemming from Boolean operators should be merged, if possible. The resulting code therefore does not necessarily mirror the structure of the if statement directly, as can be seen from the code in Pattern5. We must conclude that code generation for Boolean expressions differs in some aspects from that for arithmetic expressions. The example of Pattern 5 is also used to exhibit the code resulting from the standard procedures INC, DEC, INCL, and EXCL. These procedures provide an opportunity to use shorter code in those cases where a single two-operant instruction suffices, i.e., when one of the arguments is identical with the destination. #################### File: Project%20Oberon%20The%20Design%20of%20an%20Operating%20System%2C%20a%20Compiler%2C%20and%20a%20Computer%20-%20Niklaus%20Wirth%2C%20J%C3%BCrg%20Gutknecht%20%28PDF%29%20PO.Applications.pdf Page: 19 Context: ``` # MODULE Pattern5; VAR INTEGER: s; SET: 0, 4; BEGIN IF n = 0 THEN LDR R0 S8 ... CMP R0 R0 BNE ... INC(n) LDR R0 S8 ... ADD R0 R0 1 STR R0 S0 END; IF (n >= 0) AND (n < 100) THEN LDR R8 S8 ... LDR R0 S0 (n) CMP R0 R0 BLT 6 LDR R0 S8 CMP R0 R0 100 BGE 3 DEC(n) LDR R0 S8 ... SUB R0 R0 1 STR R0 0 END; IF (ODD(n)) OR (n IN s) THEN LDR S8 R0 ... LDR R0 S0 (n) AND R0 R0 BNE 5 LDR R0 S8 4 (s) LDR R1 S8 0 ADD R1 R1 1 ROL R0 R1 BPL 2 STR R0 S0 n = -1000 MOV R0 R0 -1000 STR R0 S0 END; IF n < 0 THEN LDR R0 S8 ... LDR R0 S0 CMP R0 R0 BGE 3 MOV R0 R0 {} STR R0 S8 4 B 17 END; ELSIF n < 10 THEN LDR R0 S8 ... LDR R0 S0 CMP R0 R0 10 BGE 3 MOV R0 R1 1 STR R0 S8 4 B 18 END; ELSIF n < 100 THEN LDR R0 S8 ... LDR R0 S0 CMP R0 R0 100 BGE 3 MOV R0 R0 2 STR R0 S8 4 B 19 ELSE MOV R0 R4 ... LDR R0 S8 ... STR R0 S8 4 END; END Pattern5. # Pattern 6: While and repeat statements. ``` #################### File: Project%20Oberon%20The%20Design%20of%20an%20Operating%20System%2C%20a%20Compiler%2C%20and%20a%20Computer%20-%20Niklaus%20Wirth%2C%20J%C3%BCrg%20Gutknecht%20%28PDF%29%20PO.Applications.pdf Page: 20 Context: ``` MODULE Pattern6; VAR i: INTEGER; BEGIN i := 0; WHILE i < 10 DO LDR R8 0 LDR R8 0 CMP R0 10 BGE 4 LDR R8 0 ADD R0 R2 STR R8 0 END; REPEAT i := i - 1 LDRB R8 0 LDR R0 R5 SUB R0 R1 STR R0 0 UNTIL i = 0 CMP R0 0 BNE -7 END Pattern6; Pattern 7: For statements. MODULE Pattern7; VAR i, m, n: INTEGER; BEGIN FOR i := 0 TO n - 1 DO LDR R1 S1 SUB R1 R1 CMP LNK R0 R1 BGT 7 STR R0 S8 4 LSL R0 R1 STR R0 S4 LDR R8 0 ADD R0 R1 END; B -11 END Pattern7; Pattern 8: Proper procedures. MODULE Pattern8; VAR z: INTEGER; PROCEDURE P(x: INTEGER; VAR y: INTEGER); BEGIN SUB SP SP 16 // adjust SP STR LNK SP 0 // push ret addr STR R0 SP 4 // push x STR R1 SP 8 // push @y z := x; LDR R0 SP 4 // x STR R0 SP 12 // z y := z; LDR R0 SP 12 // z END Pattern8; ``` #################### File: Project%20Oberon%20The%20Design%20of%20an%20Operating%20System%2C%20a%20Compiler%2C%20and%20a%20Computer%20-%20Niklaus%20Wirth%2C%20J%C3%BCrg%20Gutknecht%20%28PDF%29%20PO.Applications.pdf Page: 21 Context: # Pattern 9: Function procedures They are handled in exactly the same manner as proper procedures, except that a result is returned in register R0. If the function is called in an expression at a place where intermediate results are held in registers, these values are put onto the stack before the call, and they are restored after return (not shown here). ## MODULE Pattern9; ### VAR x: REAL; ```assembly PROCEDURE F(x: REAL); REAL; BEGIN SUB SP, SP, 8 STR LINK SP, 0 ; push ret adr STR R0 SP, 4 PUSH x IF x >= 1.0 THEN LDR R0 SP, 4 MOV R1, R0, 0xBF80H FSB R0, R0, R1 BLT 4 x := F(F(x)); LDR R0 SP, 4 BL -9 BL -10 STR R0 SP, 4 END; RETURN x; END F; ``` END Pattern9. # Pattern 10: Dynamic array parameters Dynamic array parameters are passed by loading a descriptor on the stack, regardless of whether they are value- or VAR- parameters. The descriptor consists of the actual variable's address and the array's length. (Only one-dimensional dynamic arrays are handled.) ## MODULE Pattern10; ### VAR A: ARRAY [0..12] OF INTEGER; ```assembly PROCEDURE P(var: ARRAY OF INTEGER); VAR n, i: INTEGER; BEGIN SUB SP, SP, 20 STR LINK SP, 0 STR R0 SP, 4 ; x STR SP, SP, 8.len LDR R0 SP, 12 LDR R1 SP, 8.len CMP R0, R1 BHI R12 LSL R0, R0, 2 LDR R1 SP, 4 ; x END; ``` END Pattern10. #################### File: Project%20Oberon%20The%20Design%20of%20an%20Operating%20System%2C%20a%20Compiler%2C%20and%20a%20Computer%20-%20Niklaus%20Wirth%2C%20J%C3%BCrg%20Gutknecht%20%28PDF%29%20PO.Applications.pdf Page: 22 Context: ```markdown # Code Patterns ## Pattern 11: Sets This code pattern exhibits the construction of sets. If the specified elements are constants, the set value is computed by the compiler. Otherwise, sequences of move and shift instructions are used. Since shift instructions do not check whether the shift count is within sensible bounds, the results are unpredictable if elements outside the range 0..31 are involved. ```assembly MODULE Pattern11; VAR S: SET; m, n: INTEGER; BEGIN s := (m); LDR R0 SB 4 ; m MOV R1 R0 LSL R0 R1 0 STR R0 SB 0 s := (0..n); LDR R0 SB 8 ; n MOV R1 R0 -2 LSL R0 R1 0 XOR R0 R1 0 STR R0 SB 1 s := (m..31); LDR R0 SB 4 ; m MOV R1 R0 31 MOV R1 R0 LSL R1 R1 0 STR R0 SB 0 XOR R0 R1 0 s := (m..n); LDR R0 SB 4 ; m LDR R1 SB 8 ; n MOV R2 R0 -2 LSL R2 R1 2 LSR R0 R1 1 LSL R0 R1 1 XOR R0 R1 0 STR R0 SB 0 IF n IN (2, 3, 5, 7, 11, 13) THEN MOV R0 R0 28H LDR R1 SB 8 ADD R1 R1 1 ASP R0 R1 BPL 2 END Pattern11; ``` ## END P; ADD R0 R1 R0 LDR R0 R0 0 STR R0 SP 16 LDR R0 SP 12 ; i ADD R0 R1 LDR R1 SP 8 ; x.len CMP R2 R0 R1 BLHI R12 LSL R0 R2 0 LDR R1 SP 4 ; x ADD R0 R1 R0 ADD R1 R1 5 STR R1 R0 0 ADD SP SP 20 B R15 END P; ``` #################### File: Project%20Oberon%20The%20Design%20of%20an%20Operating%20System%2C%20a%20Compiler%2C%20and%20a%20Computer%20-%20Niklaus%20Wirth%2C%20J%C3%BCrg%20Gutknecht%20%28PDF%29%20PO.Applications.pdf Page: 23 Context: ```markdown m := 1 MOV R0 R0 1 STR R0 S8 4 m END END Pattern11. # Pattern 12: Imported variables and procedures When a procedure is imported from another module, its address is unavailable to the compiler. Instead, the procedure is identified by a number obtained from the imported module's symbol file. In place of the offset, the branch instruction holds (1) the number of the imported module, (2) the number of the imported procedure, and (3) a link in the list of BL instructions calling an external procedure. This list is traversed by the linking loader, that computes the actual offset (fixup, see Chapter 6). Imported variables are also referenced by a variable's number. In general, an access required two instructions. The first loads the static base register SB from a global table with the address of that module's data section. The module number of the imported variable serves as an index. The second instruction loads the address of the variable, using the actual offset fixed up by the loader. In the following example, modules Pattern12a and Pattern12b both export a procedure and a variable. They are referenced from the importing module Pattern12c. ## MODULE Pattern12a; VAR k: INTEGER; PROCEDURE P*; BEGIN k := 1 END P; END Pattern12a. ## MODULE Pattern12b; VAR x: REAL; PROCEDURE Q*; BEGIN x := -1 END Q; END Pattern12b. ## MODULE Pattern12c; IMPORT Pattern12a, Pattern12b; VAR i: INTEGER; y: REAL; BEGIN i := Pattern12a.k; y := Pattern12b.x; END Pattern12c. # Pattern 13: Record extensions with pointers Fields of a record type R1, which is declared as an extension of a type R0, are simply appended to the fields of R0, i.e., their offsets are greater than those of the fields of R0. When a record is statically declared, its type is known by the compiler. If the record is referenced via a pointer, however, this is not the case. A pointer bound to a base type R0 may well refer to a record of an extension R1 of R0. Type tests (and type guards) allow to test for the actual type. This requires that a type can be identified at the time of program execution. Because the language defines name equivalence instead of structural equivalence of types, a type may be identified by a number. We use the address of a unique type descriptor for this purpose. ``` #################### File: Project%20Oberon%20The%20Design%20of%20an%20Operating%20System%2C%20a%20Compiler%2C%20and%20a%20Computer%20-%20Niklaus%20Wirth%2C%20J%C3%BCrg%20Gutknecht%20%28PDF%29%20PO.Applications.pdf Page: 24 Context: Therefore, type tests consist of a simple address comparison which is very fast. Type descriptors are stored in the module's area for data. Their address is called `type tag`. The tag of a (dynamically allocated) variable is stored as a prefix to its record (with offset -8). A type descriptor contains - in addition to information stored for use by the garbage collector - a table of tags of all base types. If, for instance, a type `R2` is an extension of `R1` which is an extension of `R0`, the descriptor of `R2` contains the tags of `R1` and `R0` as shown in Fig. 12.4. The table has a fixed number of 3 entries. ![Figure 12.4 Type descriptors](#) A type test of the form `p IS T` then consists of a comparison of the type tag of `p` at address `p-8` with the tag held in the descriptor of `T` at the extension level of the type of `p`. A type guard `p(T)` is synonymous to the statement: ``` IF ~ (p IS T) THEN abort END ``` The following example features 3 record types with associated pointer types, and hence also 3 type descriptors. Each descriptor is 5 words long. Their addresses, and therefore their tags, are 0, 20, and 40 respectively. ``` 20 00000020 00014006 FFFFFFFF FFFFFFFF FFFFFFFF 40 00000020 00014005 00028001 FFFFFFFF FFFFFFFF ``` ### MODULE Pattern13; ``` PO = POINTER TO R0; P1 = POINTER TO R1; P2 = POINTER TO R2; R0 = RECORD R0 : INTEGER END; R1 = RECORD R1 : INTEGER END; R2 = RECORD R2 : INTEGER END; VAR p0: P0; p1: P1; p2: P2; BEGIN p0.x := 0; p1.y := 1; p(P0).y := 3; LDR R0 S0 60 ; p0 LDR R1 S0 60 ; tag(p0) LDR R1 R1 4 ADD R2 S2 20 ; TD P1 CMP R2 R2 R1 ``` #################### File: Project%20Oberon%20The%20Design%20of%20an%20Operating%20System%2C%20a%20Compiler%2C%20and%20a%20Computer%20-%20Niklaus%20Wirth%2C%20J%C3%BCrg%20Gutknecht%20%28PDF%29%20PO.Applications.pdf Page: 25 Context: # Pattern 14: Record extensions as VAR parameters Records occurring as VAR-parameters may also require a type test at program execution time. This is because VAR-parameters effectively constitute hidden pointers. Type tests and type guards on VAR-parameters are handled in the same way as for variables referenced via pointers, with a slight difference, however. Statically declared record variables may be used as actual parameters, and they are not prefixed by a type tag. Therefore, the tag has to be supplied together with the variable's address when the procedure is called, i.e., when the actual parameter is established. Record structured VAR-parameters therefore consist of address and type tag. This is similar to dynamic array descriptors consisting of address and length. ```assembly 00000020 FFFFFFFF FFFFFFFF FFFFFFFF FFFFFFFF 00000020 00014006 FFFFFFFF FFFFFFFF FFFFFFFF ``` ## MODULE Pattern14; ### TYPE R0 = RECORD (a, b, c: INTEGER) ; VAR r0: R0; r1: R1; ### PROCEDURE P(VAR r: R0); BEGIN r.a := 1; LDR R1 SP 4 ; STR R0 R1 0 ; r(r1).d := 2; LDR R0 R8 tag(r) ; ADD R1 R8 R1 ; CMP R1 R12 R1 ; BLNE R12 ; MOV R0 R0 2 ; LDR R1 SP 4 ; STR R0 R1 12 ; END P; ... BEGIN P(r0); ADD R0 SB 40 r0 ; ADD R1 SB 0 tag(R0) ; BL P END; #################### File: Project%20Oberon%20The%20Design%20of%20an%20Operating%20System%2C%20a%20Compiler%2C%20and%20a%20Computer%20-%20Niklaus%20Wirth%2C%20J%C3%BCrg%20Gutknecht%20%28PDF%29%20PO.Applications.pdf Page: 26 Context: ```markdown # Pattern 15: Array assignments and strings. ## MODULE Pattern15: ### VAR s0: ARRAY 32 OF CHAR; ### PROCEDURE P(k: ARRAY OF CHAR); END P; BEGIN s0 := "ABCDEF"; ADD R0, SB, 0 @s0 ADD R1, SB, 64 @"ABCDEF" LDR R2, R1, 0 ADD R1, R1, 4 STR R2, R0, 0 ADD R0, 0 ASR R2, R2, 24 ; test for 0X BNE = s0 := s1; ADD R0, SB, 0 @s0 ADD R1, SB, 32 @s1 MOV R2, R0, 8 len LDR R3, R1, 0 ADD R1, R1, 4 STR R3, R0, 0 ADD R0, 4 SUB R2, R2, 1 BNE = P(1); ADD R0, SB, 32 @s1 MOV R1, 0, 32 len BL -38 P P("012345"); ADD R0, SB, 72 @"012345" MOV R1, 0, 7 ; len (incl 0X) BL -42 P P("%"); ADD R0, SB, 80 @"%" MOV R1, 0, 2 len BL -46 P END Pattern15. # Pattern 16: Predeclared procedures. ## MODULE Pattern16: ### VAR m: INTEGER; a: REAL; s: ARRAY 10 OF INTEGER; b: ARRAY 16 OF CHAR; BEGIN INC(m); LDR R1, R0, 0 @m ADD R1, R1, 1 STR R1, R0, 0 DEC(m, 10); ADD R0, SB, 4 @n LDR R1, R0, 0 SUB R1, R1, 10 STR R1, R0, 0 INCL(u, 3); ADD R0, SB, 12 @u LDR R1, R0, 0 OR R1, R1, 8 STR R1, R0, 0 {3} ``` #################### File: Project%20Oberon%20The%20Design%20of%20an%20Operating%20System%2C%20a%20Compiler%2C%20and%20a%20Computer%20-%20Niklaus%20Wirth%2C%20J%C3%BCrg%20Gutknecht%20%28PDF%29%20PO.Applications.pdf Page: 27 Context: ``` EXCL(u, 7); ADD R0 SB 12 @u LDR R1 R0 AND R1 R1 -129 ;[-7] STR R1 R0 ASSERT(m < n); LDR R1 SB 8 CMP R0 R1 BLGE R12 UNPK(x, n); LDR R0 SB 8 x ASR R1 R0 23 SUB R1 R1 127 STR R1 SB 44 n LSL R1 R1 23 STR R0 R0 R1 STR R0 SB 8 x LDR R0 SB 8 LDR R1 SB 4 n LSL R1 R1 23 ADD R0 R0 R1 STR R0 SB 8 x s := '0123456789'; ADD R1 SB 96 @s ADD R1 SB 128 adr of string LDB R2 R1 0 ADD R1 R1 4 STRB R2 R0 0 ADD R0 R0 4 ASR R2 R2 24 BNE = 0 BNE = IF s < t THEN ADD R0 SB 96 @s ADD R1 SB 112 @t LDB R2 R0 0 ADD R0 R1 1 LDB R3 R1 0 ADD R1 R1 1 CMP R4 R2 R3 BNE = 2 CMP R4 R2 0 BNE = BGE = 3 m := -1; MOV R0 R1 STR R0 SB 0 m END END Pattern16. Pattern 17: Predefined functions. MODULE Pattern17; VAR m: INTEGER; x, y: REAL; b: BOOLEAN; ch: CHAR; BEGIN n := ABS(m); LDR R0 SB 0 m CMP R0 R0 BGE 2 MOV R1 R0 SUB R0 R1 R0 STR R0 SB 8 n LSL R0 R0 1 x y := ABS(x); LDR R0 SB 8 x LSL R0 R0 1 ``` #################### File: Project%20Oberon%20The%20Design%20of%20an%20Operating%20System%2C%20a%20Compiler%2C%20and%20a%20Computer%20-%20Niklaus%20Wirth%2C%20J%C3%BCrg%20Gutknecht%20%28PDF%29%20PO.Applications.pdf Page: 28 Context: ``` ROR R0, R1 STR R0, SB 12 y b := ODD(n); LDR R0, SB 4 n AND R0, R1 BEQ 2 MOV R0, R1 B 1 MOV R0, 0 STS R0, SB 16 b n := ORD(ch); LDB R0, SB 17 ch STR R0, SB 4 n n := FLOOR(x); LDR R0, SB 4 x MOV R1, R0, 480H FAD R0, R1 STR R0, SB 1 floor STR R0, SB 4 n y := FLT(m); LDR R0, SB 0 m MOV R1, R0, 480H FAD R0, R1 float STR R0, SB 12 y n := LSL(m, 3); LSL R0, SB 4 n STR R0, SB 4 n n := ASR(m, 8); LDR R0, SB 0 8 ASR R0, SB 4 STR R0, SB 4 n m := ROR(m, n); LDR R1, SB 4 LDR R0, R0, R1 ROR R0, R1 STR R0, SB 0 END Pattern17 # 12.3. Internal data structures and module interfaces ## 12.3.1. Data structures In Section 12.1 it was explained that declarations inherently constitute context-dependence of the translation process. Although parsing still proceeds on the basis of a context-free syntax and relies on contextual information only in a few isolated instances, information provided by declarations affects the generated code significantly. During the processing of declarations, their information is transferred into the “symbol table”, a data structure of considerable complexity, from where it is retrieved for the generation of code. This dynamic data structure is defined in module ORB in terms of two record types called `Object` and `Struct`. These types prevail over all modules with the exception of the scanner. They are therefore explained before further details of the compiler are discussed (see module ORB below). For each declared identifier an instance of type `Object` is generated. The record holds the identifier and the properties associated with the identifier given in its declaration. Since Oberon is a statically typed language, every object has a type. It is represented in the record by its type field, which is a pointer to a record of type `Struct`. Since many objects may be of the same type, it is appropriate to record the type’s attributes only once and to refer to them via a pointer. The properties of type `Struct` will be discussed below. The kind of object which a table entry represents is indicated by the field class. Its values are denoted by declared integer constants: `Var` indicates that the entry describes a variable, `Con` a constant, `Fct` a record field, `Par` a VAR-parameter, and `Proc` a procedure. Different kinds of entries carry different attributes. A variable or a parameter carries an address, a constant has a value, a record field has an offset, and a procedure has an entry address, a list of parameters, and a result type. For each class the introduction of an extended record type would seem advisable. This was not done, however, for three reasons. First, the compiler was first formulated in (a subset of) 28 ``` #################### File: Project%20Oberon%20The%20Design%20of%20an%20Operating%20System%2C%20a%20Compiler%2C%20and%20a%20Computer%20-%20Niklaus%20Wirth%2C%20J%C3%BCrg%20Gutknecht%20%28PDF%29%20PO.Applications.pdf Page: 29 Context: Module-2 which does not feature type extension. Second, not making use of type extensions would make it simpler to translate the compiler into other languages for porting the language to other computers. And third, all extensions were known at the time the compiler was planned. Hence, extensibility provided no argument for the introduction of a considerable variety of types. The simplest solution lies in using the multi-purpose fields for and disc for class-specific attributes. For example, val holds an address for variables, parameters, and procedures, an offset for record fields, and a value for constants. The definition of a type yields a record of type `Struct`, regardless of whether it occurs within a type declaration, in which case also a record of type `Object` (class = `Typ`) is generated, or in a variable declaration, in which case the type remains anonymous. All types are characterized by a form and a size. A type is either a basic type or a constructed type. In the latter case, it refers to one or more other types. Constructed types are arrays, records, pointers, and procedural types. The attribute `form` refers to this classification. Its value is an integer. Just as different object classes are characterized by different attributes, different forms have different attributes. Again, the introduction of extensions of type `Struct` was avoided. Instead, some of the fields of type `Struct` remain unused in some cases, such as for basic types, and others are used for form-specific attributes. For example, the attribute `base` refers to the element type in the case of an array, to the result type in the case of a procedural type, to the type to which a pointer is bound, or to the base type of a (extended) record type. The attribute `disc` refers to the parameter list in the case of a procedural type, or to the list of fields in the case of a record type. As an example, consider the following declarations. The corresponding data structure is shown in Fig. 12.5. For details, the reader is referred to the program listing of module `ORB` and the respective explanations. ``` CONST N = 100; TYPE Ptr = POINTER TO Rec; Rec = RECORD n: INTEGER; p, q: Ptr; END; VAR k: INTEGER; a: ARRAY [0..N-1] OF INTEGER; PROCEDURE P(x: INTEGER): INTEGER; ``` #################### File: Project%20Oberon%20The%20Design%20of%20an%20Operating%20System%2C%20a%20Compiler%2C%20and%20a%20Computer%20-%20Niklaus%20Wirth%2C%20J%C3%BCrg%20Gutknecht%20%28PDF%29%20PO.Applications.pdf Page: 30 Context: # Figure 12.5. Representation of declarations | Object | Type | |-----------|------------------| | name | class | | val | type | | next | desc | | | nofpa | | | len | | N | Con | |----------|-----| | 100 | | | Ptr | Typ | |-----|---------| | | Pointer | 4 | | Rec | Typ | | | Record | 12 | | | NIL | | 0 | n | Fid | |----|----|-----| | 0 | p | Fid | | 4 | q | Fid | | 8 | | | k | Var | |----|------------| | 0 | a | Var | | 4 | Array | 400 | | NIL| | | 100| x | Var | | 4 | NIL | | P | Con | |----|-----------| | 4 | Proc | | NIL| | | 1 | | | intType | Int | 4 | Only entries representing constructed types are generated during compilation. An entry for each basic type is established by the compiler's initialization. It consists of an Object holding the standard type's identifier and a Struct indicating its form, denoted by one of the values: Byte, Bool, Char, Int, Real, or Set. The object records of the basic types are anchored in global pointer variables in module ORB (which actually should be regarded as constants). Not only are entries created upon initialization for basic types, but also for all standard procedures. Therefore, every compilation starts out with a symbol table reflecting all standard, pervasive identifiers and the objects they stand for. We now return to the subject of Objects. Whereas objects of basic classes (Const, Var, Par, Fld, Typ, SProc, SFunc, and Mod) directly reflect declared identifiers and constitute the context in which statements and expressions are compiled, compilations of expressions typically generate. #################### File: Project%20Oberon%20The%20Design%20of%20an%20Operating%20System%2C%20a%20Compiler%2C%20and%20a%20Computer%20-%20Niklaus%20Wirth%2C%20J%C3%BCrg%20Gutknecht%20%28PDF%29%20PO.Applications.pdf Page: 31 Context: # Anonymous Entities of Additional, Non-Basic Modes Such entities reflect selectors, factors, terms, etc., i.e., constituents of expressions and statements. As such, they are of a transitory nature and hence are not represented by records allocated on the heap. Instead, they are represented by record variables local to the processing procedures and are therefore allocated on the stack. Their type is called `Item` and is a slight variation of the type `Object`. Items are not referenced via pointers. Let us assume, for instance, that a term `x*y` is parsed. This implies that the operator and both factors have been parsed already. The factors `x` and `y` are represented by two variables of type `Item of Var` mode. The resulting term is again described by an item, and since the product is transitory, i.e., has significance only within the expression of which the term is a constituent, it is to be held in a temporary location, in a register. In order to express that an item is located in a register, a new, non-basic mode `Reg` is introduced. Effectively, all non-basic modes reflect the target computer's architecture, in particular its addressing modes. The more addressing modes a computer offers, the more item modes are needed to represent them. The additional item modes required by the RISC processor are declared in module `ORG`: | Reg | direct register mode | |------|-----------------------| | Reg1 | indirect register mode | | Cond | condition code mode | The use of the types `Object`, `Item`, and `Struct` for the various modes and forms, and the meaning of their attributes are explained in the following tables: ## Objects: | class | val | a | b | r | |-------|-----|-----|-----|-----| | 0 | Undef | | | | | 1 | Const | val | | | | 2 | Var | adr | base | | | 3 | Par | adr | off | | | 4 | Fld | off | off | | | 5 | Typo | TAdr | TAdr | modno | | 6 | SProc | num | | | | 7 | SFunc | num | | | | 8 | Mod | | | | ## Items: | form | nopar | len | desc | base | |------|-------|-----|------|--------| | 7 | Pointer | | base type | | 10 | ProcTyp | nopar | result type | | 12 | Array | nofel | element type | | 13 | Record | ext | lev | fields extension type | Items have an attribute called `lev` which is part of the address of the item. Positive values denote the level of nesting of the procedure in which the item is declared; `lev = 0` implies a global object. Negative values indicate that the object is imported from the module with number `-lev`. The three types `Object`, `Item`, and `Struct` are defined in module `ORB`, which also contains procedures for accessing the symbol table. ## 12.3.2. Module Interfaces (Ensure any additional information present in the reference image is filled here.) #################### File: Project%20Oberon%20The%20Design%20of%20an%20Operating%20System%2C%20a%20Compiler%2C%20and%20a%20Computer%20-%20Niklaus%20Wirth%2C%20J%C3%BCrg%20Gutknecht%20%28PDF%29%20PO.Applications.pdf Page: 32 Context: Before embarking on a presentation of the compiler's main module, the parser, an overview of its remaining modules is given in the form of their interfaces. The reader is invited to refer to them when studying the parser. The interface of the scanner module `ORS` is simple. It defines the numeric values of all symbols, but its chief constituent is procedure `Get`. Each call yields the next symbol from the source text, identified by an integer. Global variables represent attributes of the read symbol in certain cases. If a number was read, `val` or `nal` hold its numeric value. If an identifier or a string was read, `str` holds the ASCII values of the characters read. Procedure `Mark` serves to generate a diagnostic output indicating a brief diagnostic and the scanner's current position in the source text. This procedure is located in the scanner because only the scanner has access to its current position. `Mark` is called from all other modules. ```pascal IMPORT ORS; (* "Scanner" *) TYPE Ident = ARRAY 32 OF CHAR; VAR len: INTEGER; rval: REAL; id: Ident; str: ARRAY 256 OF CHAR; errn: BOOLEAN; PROCEDURE Mark(msg: ARRAY OF CHAR); PROCEDURE Get(VAR sym: INTEGER); PROCEDURE Init(source: Texts.Text; pos: INTEGER); END ORS. ``` Module `ORB` defines the basic data structures representing declared objects and their types. It also contains procedures for accessing these structures. `NewObj` serves to insert a new identifier, and it returns a pointer to the allocated object. `ThisObj` returns the pointer to the object whose name equals the global scanner variable `ORS.id`. `ThisImport` and `ThisField` deliver imported objects and record fields with names equal to `ORS.id`. Procedure `Import` serves to read the specified symbol file and to enter its identifier in the symbol table (`class = Mod`). Finally, `Export` generates the symbol file of the compiled module, containing descriptions of all objects and structures marked for export. ```pascal DEFINITION ORB; (* "Base table handler" *) TYPE Object = POINTER TO ObjDesc; Type = POINTER TO TypeDesc; ObjDesc = RECORD class, len, expn: INTEGER; expd: BOOLEAN; next: Object; type: Type; name: ORS.Id; val: INTEGER; END; TypeDesc = RECORD form: ref, mod: INTEGER; (* 'ref is used for import/export only' *) nofpnt: INTEGER; (* 'for records: extension level' *) len: INTEGER; (* 'for records: address of descriptor' *) desc: TypeObj; base: Type; size: INTEGER; END; VAR TopScope: Object; byteType, boolType, charType, intType, realType, setType, nilType, noType, strType: Type; ``` #################### File: Project%20Oberon%20The%20Design%20of%20an%20Operating%20System%2C%20a%20Compiler%2C%20and%20a%20Computer%20-%20Niklaus%20Wirth%2C%20J%C3%BCrg%20Gutknecht%20%28PDF%29%20PO.Applications.pdf Page: 33 Context: ``` # PROCEDURE Init; # PROCEDURE Close; # PROCEDURE NewObj (VAR obj: Object; id: ORS.Ident; class: INTEGER); # PROCEDURE ThisObj(): Object; # PROCEDURE ThinObject (mod: Object): Object; # PROCEDURE Finished (rec: Type): Object; # PROCEDURE OpenScope; # PROCEDURE CloseScope; # PROCEDURE Import (VAR mod: modid: ORS.Ident); # PROCEDURE Export (VAR mod: ORS.Ident); # END ORG. VAR newSF: BOOLEAN; VAR key: INTEGER; Module ORG contains the procedures for code generation. The names of these procedures indicate the respective constructs for which code is to be produced. Note that an individual code generator procedure is provided for every standard, predefined procedure. This is necessary because they generate in-line code. ## DEFINITON ORG; CONST WordSize = 4; TYPE Item = RECORD mode: INTEGER; typ: ORB.Type; a, b: INTEGER; rdb: BOOLEAN; (* "read only" *) END; VAR x: INTEGER; ### PROCEDURES - PROCEDURE MakeConstItem(VAR x: Item; typ: ORB.Type; var: INTEGER); - PROCEDURE MakeRealItem(VAR x: Item; val: REAL); - PROCEDURE MakeStringItem(VAR x: Item; len: INTEGER); - PROCEDURE MakeWrite(VAR x: Item; var: ORB.Object; curlev: INTEGER); - PROCEDURE Field(VAR x: Item; y: Object); (* x <- x.y *) - PROCEDURE Index(VAR x: VAR; y: Item); (* x <- x[y] *) - PROCEDURE Ref(VAR x: Item); - PROCEDURE BufPut(VAR x: Item; VAR y: INTEGER); - PROCEDURE TypeTest(VAR x: Item; t: ORB.Type; vararg: BOOLEAN); - PROCEDURE Not(VAR x: Item); (* x <- ~x, Boolean operators *) - PROCEDURE And1(VAR x: Item); (* x <- x && x *) - PROCEDURE And(VAR x: Item); (* x <- x && x OR *) - PROCEDURE Or1(VAR x: Item); (* x <- x OR x *) - PROCEDURE Or2(VAR x: VAR; y: Item); - PROCEDURE Neg(VAR x: Item); (* x <- -x, arithmetic operators *) - PROCEDURE AddOp(VAR x: Item); (* x <- x + y *) - PROCEDURE SubOp(VAR x: VAR; y: Item); (* x <- x - y *) - PROCEDURE MulOp(VAR x: Item; y: Item); (* x <- x * y *) - PROCEDURE DivOp(VAR x: INTEGER; VAR y: Item); (* x <- x / y *) - PROCEDURE RealOp(VAR x: INTEGER; VAR y: Item); (* x <- x op y *) - PROCEDURE Singleton(VAR x: Item); (* x <- x, set operators *) - PROCEDURE Set(VAR x: VAR; y: Item); (* x <- x ∪ y *) - PROCEDURE Insert(VAR x: Item; y: Item); (* x <- x + y *) - PROCEDURE SetOp(VAR op: INTEGER; VAR x: VAR; y: Item); (* x <- x op y *) - PROCEDURE IntRelation(VAR op: INTEGER; VAR x: Item); (* x <- x op y *) - PROCEDURE RealRelation(op: INTEGER; VAR y: Item); (* x <- x op y *) - PROCEDURE StringRelation(VAR x: VAR; y: Item); (* x <- x y *) - PROCEDURE StrToChar(VAR x: Item); (* "assignments" *) - PROCEDURE Store(VAR x: VAR; y: Item); (* x <- y *) - PROCEDURE StoreStr(VAR x: Item; y: Item); (* x <- y *) - PROCEDURE CopyString(VAR x: VAR; y: Item); (* from x to y *) ``` #################### File: Project%20Oberon%20The%20Design%20of%20an%20Operating%20System%2C%20a%20Compiler%2C%20and%20a%20Computer%20-%20Niklaus%20Wirth%2C%20J%C3%BCrg%20Gutknecht%20%28PDF%29%20PO.Applications.pdf Page: 34 Context: ``` PROCEDURE VarParam(VAR x: item; ftype: ORB.Type); (* parameters *) PROCEDURE ValueParam(VAR x: item); PROCEDURE OpenParam(VAR x: item); PROCEDURE StringParam(VAR x: item); PROCEDURE For(VAR x, y: item; VAR l: LONGINT); (* For Statements *) PROCEDURE For2(VAR x, y: item); PROCEDURE Here() : LONGINT; PROCEDURE Flump(VAR l: LONGINT); PROCEDURE CJump(VAR x: item); PROCEDURE BJump(VAR l: LONGINT); PROCEDURE CJump(VAR x: item; l: LONGINT); PROCEDURE RJump(VAR x: item); PROCEDURE Proc(VAR x: item; VAR l: LONGINT); PROCEDURE Call(VAR x: item; l: LONGINT); PROCEDURE Enter(paraId: LONGINT; locTable: LONGINT; int: BOOLEAN); PROCEDURE Return(form: INTEGER; VAR x: item; size: LONGINT; int: BOOLEAN); (* inline code procedures *) PROCEDURE Increment(updown: LONGINT; VAR x, y: item); PROCEDURE Include(index: LONGINT; VAR x, y: item); PROCEDURE Assert(VAR x: item); PROCEDURE New(VAR x: item); PROCEDURE Pack(VAR x: item); PROCEDURE Length(VAR x: y: item); PROCEDURE Get(VAR x: item); PROCEDURE Put(VAR x, y: item); PROCEDURE Copy(VAR x, y: item); PROCEDURE LDSPR(VAR x: item); PROCEDURE LRDP(VAR x: item); PROCEDURE LRED(VAR x: item); (* inline code functions *) PROCEDURE Abs(VAR x: item); PROCEDURE Sqrt(VAR x: item); PROCEDURE DStr(VAR x: item); PROCEDURE Floor(VAR x: item); PROCEDURE Float(VAR x: item); PROCEDURE Or(VAR x: item); PROCEDURE In(VAR x: item); PROCEDURE Shift(VAR LONGINT; VAR x, y: item); PROCEDURE ABC(VAR x: item); PROCEDURE SLC(VAR x: item); PROCEDURE UML(VAR x, y: item); PROCEDURE BVAR(VAR x: item); PROCEDURE Register(VAR x: item); PROCEDURE HVAR(VAR x: item); PROCEDURE Add(VAR x: item); PROCEDURE Open(n: INTEGER); PROCEDURE SetDataSize(id: LONGINT); PROCEDURE Header(); PROCEDURE Close(VAR mod: ORS.Ident; key, n: LONGINT); END ORG. # 12. A Parser The main module ORP constitutes the parser. Its single command **Compile** - at the end of the program listing - identifies the source text according to the Oberon command conventions. It then calls procedure **Module** with the identified source text as parameter. The command forms are: ``` #################### File: Project%20Oberon%20The%20Design%20of%20an%20Operating%20System%2C%20a%20Compiler%2C%20and%20a%20Computer%20-%20Niklaus%20Wirth%2C%20J%C3%BCrg%20Gutknecht%20%28PDF%29%20PO.Applications.pdf Page: 35 Context: # ORP Compiler ## Overview ### ORP.Compile @ The most recent selection identifies the beginning of the source text. ### ORP.Compile ^ The most recent selection identifies the name of the source file. ### ORP.Compile f0 | f1 | ... These are the names of source files. File names and the characters `@` and `^` may be followed by an option specification `/s`. Option `/s` enables the compiler to overwrite an existing symbol file, thereby invalidating clients. ## Parser Design The parser is designed according to the proven method of top-down, recursive descent parsing with a look-ahead of a single symbol. The last symbol read is represented by the global variable `sym`. Syntactic entities are mirrored by procedures of the same name. Their goal is to recognize the specified construct in the source text. The start symbol and corresponding procedure is `Module`. The principal parser procedures are shown in Fig. 12.6, which also exhibits their calling hierarchy. Loops in the diagram indicate recursion in the syntactic definition. ### Figure 12.6: Parser Procedure Hierarchy ``` Module ___________ | | Declarations StatSeq | | Type ProcDecl _______ _______ | | | | RecType ArrayType FPSection | | | FormalType ParameterList ________ | | ParamList SimpleExp ____________ | | term expression | factor | element ``` ## Rule Violations The rule of parsing strictly based on a single-symbol look-ahead and without reference to context is violated in three places. The prominent violation occurs in statements. If the first symbol of a statement is an identifier, the decision of whether an assignment or a procedure call is to be recognized is based on contextual information, namely the class of the identified object. The second violation occurs in `qualified`: if the identifier `x` preceding a period denotes a module, it is recognized together with the subsequent identifier as a qualified identifier. Otherwise, it is supposedly denotes a record variable. The third violation is made in procedure `selector`: if an identifier is followed by a left parenthesis, the decision of whether a procedure call or a type guard is to be recognized is again made on the basis of contextual information, namely the mode of the identified object. ## Error Discovery A fairly large part of the program is devoted to the discovery of errors. Not only should they be properly diagnosed, but a much more difficult requirement is that the parsing process should continue on the basis of a good guess about the structure that the text should most likely have. The parsing process must continue with some assumption and possibly after skipping a short piece of the object. #################### File: Project%20Oberon%20The%20Design%20of%20an%20Operating%20System%2C%20a%20Compiler%2C%20and%20a%20Computer%20-%20Niklaus%20Wirth%2C%20J%C3%BCrg%20Gutknecht%20%28PDF%29%20PO.Applications.pdf Page: 36 Context: # Source Text Hence, this aspect of the parser is mostly based on heuristics. Incorrect assumptions about the nature of a syntactic error lead to secondary error diagnostics. There is no way to avoid them. A reasonably good result is obtained by the fact that procedure `ORS.Mark` initiates an error report, if it is less than 10 characters ahead of the last one. Also, the language Oberon is designed with the property that most large constructs begin with a unique symbol, such as IF, WHILE, CASE, RECORD, etc. These symbols facilitate the recovery of the parsing process in the erroneous text. More problematic are open constructs which neither begin nor end with key symbols, such as types, factors, and expressions. Relying on heuristics, the source text is skipped up to the first occurrence of a symbol which may begin a construct that follows the one being parsed. The employed scheme may not be the best possible, but it yields quite acceptable results and keeps the amount of program devoted to the handling of erroneous texts within justifiable bounds. ## Type Checking Besides the parsing of text, the Parser also performs the checking for type consistency of objects. This is based on type information held in the global table, gained during the processing of declarations, which is also handled by the routines which parse. Thereby an unjustifiably large number of very short procedures is avoided. However, the strict target-computer independence of the parser is violated slightly: Information about variable allocation strategy including alignment, and about the sizes of basic types is used in the parser module. Whereas the former violation is harmless, because the allocation strategy is hardly controversial, the latter case constitutes a genuine target-dependence embodied in a number of explicitly declared constants. Mostly these constants are contained in the respective type definitions, represented by records of type `Typ` initialized by `ORB`. The following procedures allocate objects and generate elements of the symbol table: | Declarations | Object (Con), Object (Typ), Object (Var) | |---------------------|-------------------------------------------| | ProcedureDeclaration | Object (Proc) | | FormalType | Object (Var), Object (Par) | | ORB.Import | Object (Mod) | | RecordType | Object (Fid), Type (Record) | | ArrayType | Type (Array) | | ProcedureType | Type (ProcTyp) | | Type | Type (Pointer) | | FormalType | Type (Array) | ## Forward Declarations An inherently nasty subject is the treatment of forward references in a single-pass compiler. In Oberon, there are two such cases: 1. Forward declarations of procedures. They have been eliminated from the revision of the Oberon language in the year 2007 as they should be avoided if ever possible. If it is impossible, a remedy is to declare a variable of the given procedure type, and assign the procedure to be forwarded to this variable. The nastiness of procedure forward declarations originates in the necessity to specify parameters and result type in the forward declaration. These must be repeated in the actual procedure declaration, and one expects that a compiler verifies the equality (or equivalence) of the two declarations. This is a heavy burden for a case that very rarely occurs. 2. Forward declarations of pointer types also constitutes a nasty exception, but its exclusion would be difficult to justify. If in a pointer declaration the base type (to which the pointer is bound) is not found in the symbol table, a forward reference is therefore automatically assumed. An entry for the pointer type is generated anyway (see procedure Type) and an element is inserted in the list of pointer types to be fixed up. This list is headed by the global variable `pobsList`. When later in the text a declaration of a record type is encountered with the same identifier, the forward entry is recognized and the proper link is established (see procedure Declarations). The compiler must check for undefined forward references when the current declaration scope is closed. This check is performed at the end of procedure Declarations. #################### File: Project%20Oberon%20The%20Design%20of%20an%20Operating%20System%2C%20a%20Compiler%2C%20and%20a%20Computer%20-%20Niklaus%20Wirth%2C%20J%C3%BCrg%20Gutknecht%20%28PDF%29%20PO.Applications.pdf Page: 37 Context: The IF statement had been eliminated from the language in its revision of 2007. Here it reappears in the form of a case statement, whose cases are not labelled by integers, but rather by types. What formerly was written as: ```markdown IF x IS T1 THEN WITH x:T1 DO ... x ... END ELSEIF x IS T2 THEN WITH x:T2 DO ... x ... END ELSEIF ... END ``` is now written more simply and more efficiently as: ```markdown CASE x OF T1: ... x ... | T2: ... x ... | ... END ``` where T1 and T2 are extensions of the type T0 of the case variable x. Compilation of this form of case statement merges the regional type guard of the former with statements with the type test of the former if statements. This case statement represents the only case where a symbol table entry - the type of x - is modified during compilation of statements. When the end of the with statement is reached, the change must be reverted. ## 12.5 The scanner The scanner module ORS embodies the lexicographic definitions of the language, i.e., the definition of abstract symbols in terms of characters. The scanner's substance is procedure `Get`, which scans the source text and, for each call, identifies the next symbol and yields the corresponding integer code. It is most important that this process be as efficient as possible. Procedure `Get` recognizes letters indicating the presence of an identifier (or reserved word), and digits signifying the presence of a number. Also, the scanner recognizes comments and skips them. The global variable `ch` stands for the last character read. A sequence of letters and digits may either denote an identifier or a key word. In order to determine this, a search is made in a table containing all key words for each would-be identifier. This table is sorted alphabetically and according to the length of reserved words. It is initialized when the compiler is loaded. The presence of a digit signals a number. Procedure `Number` first scans the subsequent digits (and letters) and stores them in a buffer. This is necessary, because hexadecimal numbers are denoted by the postfix letter H (rather than a prefix character). The postfix letter X specifies that the digits denote a character. There exists one case in the language Oberon, where a look-ahead of a single character does not suffice to identify the next symbol. When a sequence of digits is followed by a period, this period may either be the decimal point of a real number, or it may be the first element of a range (..). Fortunately, the problem can be solved locally as follows: If, after reading digits and a period, a second period is present, the number symbol is returned, and the local variable `ch` is assigned the special value 7FX. A subsequent call of `Get` then delivers the range symbol. Otherwise the reader after the digit sequence belongs to the (real) number. ## 12.6 Searching the symbol table, and handling symbol files ### 12.6.1 The structure of the symbol table The symbol table constitutes the context in which statements and expressions are parsed. Each procedure establishes a scope of visibility of local identifiers. The records registering identifiers belonging to a scope are linked as a linear list. They are of type `Object`. Each object has a type. #################### File: Project%20Oberon%20The%20Design%20of%20an%20Operating%20System%2C%20a%20Compiler%2C%20and%20a%20Computer%20-%20Niklaus%20Wirth%2C%20J%C3%BCrg%20Gutknecht%20%28PDF%29%20PO.Applications.pdf Page: 38 Context: ```markdown Types are represented by records of type `Type`. These two types pervade the entire compiler, and they are defined as follows: ```pascal TYPE Object = POINTER TO ObjDesc; Type = POINTER TO TypeDesc; ObjDesc = RECORD class: INTEGER; exp: BOOLEAN; (* exported / read-only *) next: Object; type: Type; name: ORS.Ident; val: INTEGER END; TypeDesc = RECORD form: ref, min: INTEGER; (* ref is only used for import/export *) npl: INTEGER; (* for procedures; extension level for record's *) len: INTEGER; (* for arrays, len = 0 -> open array; for records; adr of descriptor *) desc: typb: Object; base: Type; (* for arrays, records, pointers *) size: INTEGER; (* in bytes; always multiple of 4, except for Byte, Bool and Char *) END; ``` Procedures for generating and searching the lists are contained in module `ORB`. If a new identifier is to be added, procedure `NewObj` first searches the list, and if the identifier is already present, a double definition is diagnosed. Otherwise, the new element is appended, thereby preserving the order given by the source text. Procedures, and therefore also scopes, may be nested. Each scope is represented by the list of its declared identifiers, and the list of the currently visible scopes are again connected as a list. Procedure `OpenScope` appends an element and procedure `CloseScope` removes it. The list of scopes is anchored in the global variable `topScope` and linked by the field `desc`. It is treated like a stack. It consists of elements of type `Object`, each one being the header (`class = Head`) of the list of declared entities. As an example, the procedure for searching an object (with name `ORS.id`) is shown here: ```pascal PROCEDURE ThisObj(): Object; VAR s: Object; BEGIN s := topScope; REPEAT x := s.next; WHILE (x # NIL) AND (x.name # ORS.id) DO x := x.next; s := dsc UNTIL (x # NIL) OR (s = NIL); RETURN x; END ThisObj; ``` A snapshot of a symbol table for an example with nested scopes is shown in Fig. 12.6. It is taken when the following declarations are parsed and when the statement `s` is reached. ```pascal VAR x: INTEGER; PROCEDURE P(): INTEGER; BEGIN END P; PROCEDURE Q(v: INTEGER); BEGIN PROCEDURE R(w: INTEGER); BEGIN B := END R; END; END Q; ``` ``` #################### File: Project%20Oberon%20The%20Design%20of%20an%20Operating%20System%2C%20a%20Compiler%2C%20and%20a%20Computer%20-%20Niklaus%20Wirth%2C%20J%C3%BCrg%20Gutknecht%20%28PDF%29%20PO.Applications.pdf Page: 39 Context: # 12.6.2 Symbol files A search of an identifier proceeds first through the scope list, and for each header its list of object records is scanned. This mirrors the scope rule of the language and guarantees that if several entities carry the same identifier, the most local one is selected. The linear list of objects represents the simplest implementation by far. A tree structure would in many cases be more efficient for searching and would therefore seem more recommendable. Experiments have shown, however, that the gain in speed is marginal. The reason is that the lists are typically quite short. The superiority of a tree structure becomes manifest only when a large number of global objects is declared. We emphasize that when a tree structure is used for each scope, the linear lists must still be present, because the order of declarations is sometimes relevant in interpretation, e.g., in parameter lists. Not only procedures, but also record types establish their own local scope. The list of record fields is anchored in the type record's field des, and it is searched by procedure `thisField`. If a record type `R1` is an extension of `R0`, then `R1`'s field list contains only the fields of the extension proper. The base type `R0` is referenced by the `BaseTyp` field of `R1`. Hence, a search for a field may have to proceed through the field lists of an entire sequence of record base types. The major part of module `ORB` is devoted to input and output of symbol files. A symbol file is a linearized form of an excerpt of the symbol table containing descriptions of all exported (marked) objects. All exports are declared in the global scope. Procedure `Export` traverses the list of global objects and outputs them to the symbol file. The structure of a symbol file is defined by the syntax specified below. The following terminal symbols are class and form specifiers or reference numbers for basic types with fixed values: Classes: - `Cons`: `C` = 1, `Var` = 2, `Par` = 3, `Id` = 4, `Typ` = 5 Forms: - `Byte` = 1, `Bool` = 2, `Char` = 3, `Int` = 4, `Lint` = 5, `Set` = 6 - `Pointer` = 7, `NotBy` = 9, `ProcTyp` = 10, `Array` = 12, `Record` = 13 ```markdown Symbol = null key name versionkey (object) object = (CON name type {value} [ | TYPE name type {fix} 0 | VAR name type expn]) type = ref (PTR type | ARR type len | REC type {field} [ | PRO type {param}]) ``` #################### File: Project%20Oberon%20The%20Design%20of%20an%20Operating%20System%2C%20a%20Compiler%2C%20and%20a%20Computer%20-%20Niklaus%20Wirth%2C%20J%C3%BCrg%20Gutknecht%20%28PDF%29%20PO.Applications.pdf Page: 40 Context: # Field Type Description - **field** = FLD name type offset. - **param** = (VAR | PAR) type. A procedure type description contains a parameter list. Similarly, a record type description with form specifier `Record` contains the list of field descriptions. Note that a procedure is considered as a constant of a procedure type. Objects have types, and types are referenced by pointers. These cannot be written on a file. The straightforward solution would be to use the type identifiers as they appear in the program to denote types. However, this would be rather crude and inefficient, and second, there are anonymous types, for which artificial identifiers would have to be generated. An elegant solution lies in consecutively numbering types. Whenever a type is encountered the first time, it is assigned a unique **reference number**. For this purpose, records (in the compiler) of type `Type` contain the field ref. Following the number, a description of the type is then written to the symbol file. When the type is encountered again during the traversal of the data structure, only the reference number is issued, with a negative sign. The global variable `OBR.Ref` functions as the running reference number. When reading a symbol file, a positive reference number is followed by the type’s description. A pointer to the type read is assigned to the global table `typab` with the reference number as index. When a negative reference number is read (it is not followed by a type description), then the type is identified by `typab[ref]` (see procedure in `Type`). In the following example, types are identified by their reference number (e.g. `#14`), and later referenced by this number (`'14`). ```pascal MODULE A; CONST "10 Dollar" = "$"; TYPE R = RECORD v: INTEGER; v: SET END; END; P = POINTER TO R; A = ARRAY 8 OF INTEGER; B = ARRAY 4 OF REAL; C = ARRAY 10 OF CHAR; V = ARRAY OF INTEGER; PROCEDURE O0; BEGIN END O0; PROCEDURE O1(x: INTEGER); INTEGER; BEGIN RETURN x + y END O1; END A. ``` ### Class Definitions - **class CON**: CON [14] - **class CON**: CON [16] - **class**: TYP = [14] from `REC [14]` environ =`1 extlev = 0 size = 8` | v[6] 4 [14] [0] - **class**: TYP = [15] from `REC [15]` environ = `0 size = 32` | v[60] form = `ARR [14] len = 4 size = 32 [0]` - **class**: TYP = [16] from `ERR [14]` - **class**: TYP = [17] from `ARR [14]` | `len = 8 size = 32` - **class**: TYP = [18] from `ARR [15]` | `len = 5 size = 20` | `len = 4 size = 80` - **class**: TYP = [19] form = `ARR [15]` | `len = 10 size = 320` - **class**: TYP = [20] form = `ARR [15]` | `len = 1 size = 8` - **class**: VAR [14] - **class**: CON [30] from `PRO [14]` - **class**: CON [40] from `PRO [16]` After a symbol file has been generated, it is compared with the file from a previous compilation of the same module, if one exists. Only if the two files differ and if the compiler's option is enabled, is the old file replaced by the new version. The comparison is made by comparing byte after byte without consideration of the file's structure. This somewhat crude approach was chosen because of its simplicity and yielded good results in practice. A symbol file must not contain addresses of variables or procedures. If they did, most changes in the program would result in a change of the symbol file. This must be avoided, because changes in the program result in changes in the symbol file. #################### File: Project%20Oberon%20The%20Design%20of%20an%20Operating%20System%2C%20a%20Compiler%2C%20and%20a%20Computer%20-%20Niklaus%20Wirth%2C%20J%C3%BCrg%20Gutknecht%20%28PDF%29%20PO.Applications.pdf Page: 41 Context: the implementation (rather than the interface) of a module are supposed to remain invisible to the clients. Only changes in the interface are allowed to affect changes in the symbol file, requiring recompilation of all clients. Therefore, addresses are replaced by export numbers. The variable `exno` (global in ORP) serves as running number (see ORP.Declarations and ORP.ProcedureDecl). The translation from export number to address is performed by the loader. Every code line contains a list (table) of addresses (of variables and entry points for procedures). The export number serves as index in this table to obtain the requested address. Export numbers are generated by the parser. Objects exported from some module `M1` may refer in their declaration to some other module `M0` imported by `M1`. It would be unacceptable if an import of `M1` would then also require the import of `M0`, i.e., imply the automatic reading of `M0`'s symbol file. It would trigger a chain reaction of imports that must be avoided. Fortunately, such a chain reaction can be avoided by making symbol files self-contained, i.e., by including in every symbol file the description of entities that stem from other modules. Such entities are always types. The inclusion of types imported from other modules seems simple enough to handle: type descriptions must include a reference to the module from which the type was imported. This reference is the name and key of the respective module. However, there exists one additional fact that cannot be ignored. Consider a module `M1` importing a variable `x` from a module `M0`. Let the type `T` of `x` be defined in module `M0`. Also, assume `M1` to contain a variable of type `M0.T`. Evidently, `x` and `y` are of the same type, and the compiler compiling `M2` must recognize this fact. Hence, when importing `M0` during compilation of `M1`, the imported element `T` must not only be registered in the symbol table, but it must also be recognized as being identical to `T` already imported from `M2` directly. It is rather fortunate that the language definition specifies equivalence of types on the basis of names rather than structure, because it allows type tests at execution time to be implemented by a simple address comparison. The measures to be taken to satisfy the new requirements are as follows: 1. Every type element in a symbol file is given a module number. Before a type description is emitted to the file. 2. If a type to be exported has a name and stems from another, imported module, then also the name and key of the module are attached, from which the type stems (see end of procedure ORB.OutType and end of ORB.InType). An additional detail is worth being mentioned here: Hidden pointers. We recall that individual fields of exported record types may be hidden. If marked (by an asterisk) they are exported and therefore visible in importing modules. If not marked, they are not exported and remain invisible, and evidently seem to be ominous in symbol files. However, this is a fallacy. They need to be included in symbol files, although without name, because of meta information to be provided for garbage collection. This is elucidated as follows: Assume that a module `M1` declares a global pointer variables of a type imported from module `M0`. ```pascal MODULE M0; TYPE PTR = POINTER TO Rec0; Rec0 = RECORD p1, q: PTR; ... END; END M0. MODULE M1; VAR p: M0.PTR; R: RECORD M0.Ptr; ... END; END M1. ``` Here `p` and `r` are roots of data structures that must be visited by the garbage collector. If they are not, they will not be marked, and therefore collected with disastrous and entirely unpredictable consequences. The crux is that not only exported pointers (p1, p) must be listed, but also hidden ones (p.q), although they are not accessible in module `M1`. #################### File: Project%20Oberon%20The%20Design%20of%20an%20Operating%20System%2C%20a%20Compiler%2C%20and%20a%20Computer%20-%20Niklaus%20Wirth%2C%20J%C3%BCrg%20Gutknecht%20%28PDF%29%20PO.Applications.pdf Page: 42 Context: We chose to include hidden pointers in symbol files without their names, but with their type being of the form `ORB.NIType`. This must be considered in procedure `ORG.FindPrts`, where the condition `typ.form = ORB.Pointer` must be extended to `(typ.form = ORB.Pointer OR (typ.form = ORB.NIType))`. But the story does not end here. Assume that in the example above module `M1` declares a type `Rec1` as an extension of `Mo.Reco`. This requires the generation of a type descriptor. And this descriptor must include not only field `p`, but also the hidden field `d`. This is achieved by also extending the condition `typ.form = ORB.Pointer in ORG.FindPrtFlds` to `(typ.form = ORB.Pointer OR (typ.form = ORB.NIType))`. ## 12.7 The code generator The routines for generating instructions are contained in a single module: ORG. They are fairly numerous, and therefore the interface of ORG is quite large. It is a procedural interface. This implies that there is no "intermediate code" or "intermediate data structure" between parser and code generator. This is one reason for the complexities of the code generator. The other is the regularity and simplicity of the processor architecture. In order to understand the following material, the reader is supposed to be familiar with this architecture (Appendix 2) and the generated code patterns for individual language constructs (Section 12.2). A distinguishing feature of this compiler is that parsing proceeds top-down according to the principle of recursive descent in the parsing tree. This implies that for every syntactic construct a specific procedure is called. It carries the same name as the construct. It also implies that properties of the parsed construct can be represented by parameters of the parsing procedures. Consider, for example, the construct of simple expression: ``` SimpleExpression = term { "+" term }. ``` The corresponding parsing procedure is: ``` PROCEDURE SimpleExpression(VAR x: Item); VAR y: Item; BEGIN itemx := ""; WHILE sym = plus DO ORS.Get(sym); term(y); ORG.AddOp(x, y) END END SimpleExpression ``` The generating procedure `AddOp` receives two parameters representing the operands, and returns the result through the first parameter. This scheme carries the invaluable advantage of using operands efficiently allocated on the stack rather than dynamically allocated on the heap, subject to automatic storage retrieval (garbage collection). Here the processed operands quickly disappear from the stack upon exit from the parser procedure. The parameters representing syntactic constructs are of type `Item` defined in ORG. This data type is rather similar to the type `Object` in ORB. After all, it serves the same purpose; but it represents internal items rather than declared objects. ``` TYPE Item = RECORD mode: INTEGER; type: ORB.TYPE; a: b: INTEGER; rd: BOOLEAN; (* “read only” *) END ``` The attribute class of `Object` is renamed into `Item`. In fact, in some sense different classes evoke different (corresponding) addressing modes as featured by the processor architecture. According to the architecture, additional modes may have to be introduced. Thanks to the simplicity of RISC, only three are needed: - `Reg = 10`: The item `x` is located in register `x.r` - `Reg1 = 11`: The item `x` is addressed indirectly through register `x.r` plus offset `x.a` - `Cond = 12`: The item is represented by the condition bit registers. #################### File: Project%20Oberon%20The%20Design%20of%20an%20Operating%20System%2C%20a%20Compiler%2C%20and%20a%20Computer%20-%20Niklaus%20Wirth%2C%20J%C3%BCrg%20Gutknecht%20%28PDF%29%20PO.Applications.pdf Page: 43 Context: # Instructions Instructions are emitted sequentially and emitted by the four procedures `Put0`, `Put1`, `Put2`, `Put3`. They directly correspond to the instruction formats of the RISC processor (see Chapter 11). The instructions are stored in the array code and the compiler variable `pc` serves as running index. ``` PROCEDURE Put0(op, a, b: INTEGER); // format F0 PROCEDURE Put1(op, a, b: INTEGER); // format F1 PROCEDURE Put2(op, a, b: OFF INTEGER); // format F2 PROCEDURE Put3(op, cond: OFF INTEGER); // format F3 ``` ## 12.7.1. Expressions Expressions consist of operands and operators. They are evaluated and have a value. First, a number of make-procedures transform objects into items (see Section 12.3.2). The principal one is `MakeItem`. Typical objects are variables (class, mode = `Var`). Global variables are addressed with base register `SB` (x7 = 13), local variables with the stack pointer `SP` (x7 = 14). VAR-parameters are addressed indirectly; the address is on the stack (class, mode = `Par`, `Ind`). `x.a` is the offset from the stack pointer. Before an operator can be applied to operands, these must first be transferred (loaded) into registers. This is because the RISC performs operations only on registers. The loading is achieved by procedure `load` (and `loadAdr`) in `ORG`. The resulting mode is `Reg`. In allocating registers, a strict stack principle is used, starting with `R0` to `R11`. This is certainly not an optimal strategy and provides ample room for improvement (usually called optimization). The compiler variable `RH` indicates the next free register (top of register stack). Base address `SB` is, as the name suggests, static. But this holds only within a module. It implies that on every transfer to a procedure in another module, the static base must be adjusted. The simplest way is to load `SB` before every external call, and to restore it to its old value after return from the procedure. We chose a different strategy: loading on demand (see below: global variables). If a variable is indexed, has a field selector, is dereferenced, or has a type guard, this is detected in the parser by procedure `selector`. It calls generators `Index`, `Field`, `DeRef`, or `TypeTest` accordingly (see Section 12.3.2 and patterns 1-4 in Section 12.2). These procedures cause item modes to change as follows: | mode transition of x | instructions emitted | construct | |----------------------|--------------------------------|-----------------------------------| | `Index(x, y)` | (y is loaded into x.y) | `ADD y, SP, y` | | | | field designate, add offset to x.a | | `Par -> Reg` | `LDR RH, SP, x.a` | array variable | | | `ADD y, RH, y` | array parameter | | `Reg -> Reg` | `ADD x, x.r, y` | indexed array | ### 2. Field(x, y) (y.mode = Field, y.a = a field offset) | Transition | From | To | Construction | |----------------|---------------|----------------|----------------------------------| | `Var -> Var` | none | Field designator, add offset to x.a | | | `Reg1 -> Reg` | none | add field offset to x.a | | | `Par -> Reg` | none | add field offset to x.b | | ### 3. DeRef(x) | Transition | From | To | Construction | |----------------|-------------|------------------------|---------------------------| | `Var -> Reg` | `LDR RH, SP, x.a` | dereferenced x^ | | | `Par -> Reg` | `LDR RH, SP, x.a` | dereferenced parameter x^ | A fairly large number of procedures then deal with individual operators. Specifically, they are `Not`, `And1`, `And2`, `Or1`, `Or2` for Boolean operators, `Neg`, `AddOp`, `MulOp`, `DivOp` for operations on integers, `RealOp` for operations on real numbers, and `Singleton`, `Set`, `In`, and `SetOp` for operations. #################### File: Project%20Oberon%20The%20Design%20of%20an%20Operating%20System%2C%20a%20Compiler%2C%20and%20a%20Computer%20-%20Niklaus%20Wirth%2C%20J%C3%BCrg%20Gutknecht%20%28PDF%29%20PO.Applications.pdf Page: 44 Context: # 12.7.2 Relations RISC does not feature any compare instruction. Instead, subtraction is used, because an implicit comparison with 0 is performed along with any arithmetic (or load) instruction. Instead of \( x < y \) we use \( x - y \). This is possible because, in addition to the computed difference deposited in a register, also the result of the comparison is deposited in the condition flags \( N \) (difference negative) and \( Z \) (difference zero). Relations therefore yield a result item \( x \) with mode `Cond.x < y (rel(jumps))` identifies the relation. Branch instructions (jumps) are executed conditionally depending on these flags. The value \( x.r \) is then used when generating branch instructions. For example, the relation \( x < y \) is translated simply into: ``` LDR R0, SP, x LDR R1, SP, y CMP R0, R0, R1 ``` and the resulting item mode is `x.mode = Cond, x.r = "less"`. (The mnemonic CMP is synonymous with SUB). More about relations and Boolean expressions will be explained in Section 12.7.6. ## 12.7.3 Set operations The type `SET` represents sets of small integers in the range from 0 to 31. Bit \( i \) signals that \( i \) is an element of the set. This is a convenient representation because the logical instructions directly mirror the set operations: AND implements set intersection, OR set union, and XOR the symmetric set difference. This representation also allows a simple and efficient implementation of membership tests. The instructions for the expression \( \text{m IN s} \) is generated by procedure `In`. Assuming the value in register R0, and the set \( s \) in R1, we obtain: ``` ADD R0, R0, 1 ROR R1, R1, R0 // rotate s by i+1 position, the relevant bit moving to the sign bit ``` The resulting item mode is `Cond.x = r - "minus"`. Of some interest are the procedures for generating sets, i.e. for processing \( \{m\}, \{m .. n\} \), and \( \{m, n\} \), where \( m, n \) are integer expressions. We start with \( \{m\} \). It is generated by procedure `Singleton` using a shift instruction. Assuming \( m \) in R0, the resulting code is: ``` MOV R1, 0, 1 LSL R0, R1, R0 // shift 1 by m bits positions to the left ``` Somewhat more sophisticated is the generation of \( \{m .. n\} \) by procedure `Set`. Assuming \( m \) in R0, and \( n \) in R1, the resulting code is: ``` MOV R0, 0, 2 LSL R1, R2, R1 // shift -2 by n bits positions to the left MOV R0, 2, -1 LSL R0, R2, R0 // shift -1 by m bits positions to the left XOR R0, R0, R1 ``` The set \( \{m, n\} \) is generated as the union of \( \{m\} \) and \( \{n\} \). If any of the element values is a constant, several possibilities of code improvement are possible. For details, the reader is referred to the source code of ORG. ## 12.7.4 Assignments #################### File: Project%20Oberon%20The%20Design%20of%20an%20Operating%20System%2C%20a%20Compiler%2C%20and%20a%20Computer%20-%20Niklaus%20Wirth%2C%20J%C3%BCrg%20Gutknecht%20%28PDF%29%20PO.Applications.pdf Page: 45 Context: ``` Statements have an effect, but no result like expressions. Statements are executed, not evaluated. Assignments alter the value of variables through store instructions. The computation of the address of the affected variable follows the same scheme as for loading. The value to be assigned must be in a register. Assignments of arrays (and records) are an exceptional case in so far as they are performed not by a single store instruction, but by a repetition. Consider `y = x`, where `x` and `y` are both arrays of `n` integers. Assuming that the address of `y` is in register `R0`, that of `x` in `R1`, and the value in `R2`. Then the resulting code is: ``` LDR R3, R1, 0 ; source ADD R1, R1, 4 ; increment index STR R3, R0, 0 ; destination ADD R0, R0, 4 ; increment destination SUB R2, R2, 1 ; counter BNE L2 ; branch if not equal ``` ## 12.7.5 Conditional and repetitive statements These statements are implemented using branch instructions (jumps) as shown in Section 12.2, Patterns 5 - 7. In all repetitive statements, backward jumps occur. Here, at the point of return the value of the global variable `ORG.pc` is saved in a local (`l`) variable of the involved parsing procedure. It is retrieved when the backward jump is emitted. We note that branch instructions use a displacement rather than an absolute destination address. It is the difference between the branch instruction and the destination of the jump. A difficulty, however, arises in the case of forward jumps, a difficulty inherent in all single-pass compilers: When the branch is issued, its destination is still unknown. It follows that the branch displacement must be later resolved when it becomes known, when the destination is reached. This is called a `fixup`. Here the method of fixup lists is used. The place of the instruction with still unknown destination is held in a variable `L` for the respective parsing procedure. If several branches have the same destination, `L` is the heading of a list of the instructions to be fixed up, with its links placed in the instructions themselves in the place of the eventual jump displacement. This shows how to fix a statement by an excerpt of `ORP.StaSequence` with local variable `L0`: ``` ELSEIF sym = ORS.IF THEN ORS.Get(sym); expression(x); ORG.FJump(x); StatSequence: L0 := 0; WHILE sym = ORS.ELSE DO ORS.Get(sym); ORG.FJump(L0); ORG.Fixup(x); expression(x); END; IF sym = ORS.Else THEN ORS.Get(sym); ORG.FJump(L0); ORG.Fixup(x); StatSequence ELSE ORG.Fixup(x); END; ORG.FixLink(L0); ``` ### where in module ORG: ``` PROCEDURE FJump(VAR x: Item); (* conditional forward jump *) BEGIN IF x.mode <> Cond THEN loadCond(x) END; PUJ3(BG, negated(x), x.a); FLinkLink(x); x.a := pc-1; END FJump; PROCEDURE JFJump(VAR L: LONGINT); (* unconditional forward jump *) BEGIN PUJ3(BC, 7); L := L - 1; END FJump; PROCEDURE fix(at: VAR LONGINT); BEGIN code[at] := code[at] DIV C24 * C24 + (with MOD C24); END fix; ``` ``` #################### File: Project%20Oberon%20The%20Design%20of%20an%20Operating%20System%2C%20a%20Compiler%2C%20and%20a%20Computer%20-%20Niklaus%20Wirth%2C%20J%C3%BCrg%20Gutknecht%20%28PDF%29%20PO.Applications.pdf Page: 46 Context: ``` PROCEDURE FixLink(L: LONGINT); VAR L1: LONGINT; BEGIN InvalS := 0; WHILE L <> 0 DO L1 := code[L] MOD 40000; FixLink(L1); L := L1 END END FixLink; PROCEDURE Fixup(VAR x: Item); BEGIN FixLink(x.a) END Fixup; In while-, repeat-, and for statements essentially the same technique is used with the support of the identical procedures in ORG. ## 12.7. Boolean expressions In the case of arithmetic expressions, our compilation scheme results in a conversion from infix to postfix notation (x+y ↔=> x+y). This is not applicable for Boolean expressions, because the operators & (AND) and OR are defined as follows: - x & y → if x then y else FALSE - x OR y → if x then TRUE else y This entails that depending on the value of x, y must not be evaluated. As a consequence, jumps may have to be taken across the code for y. Therefore, the same technique of conditional evaluation must be used for conditional statements. In the case of an expression x & y (or x OR y), procedure ORG.And resp. ORG.OR1 must be called just after parsing x (see ORP.Term resp. ORP.SimpleExpression). Only after parsing also can the generators ORG.And resp. ORG.Or2 be called, providing the necessary fixups or forward jumps. PROCEDURE And1(VAR x: Item); (* x := x & a *) BEGIN IF x.mode & Cond THEN loadCond(y) END; Buf3d.Reg, negated(x.a); x.a := pc; FixLink(x.b); x.b := 0 END And1; PROCEDURE And2(VAR x: Item); BEGIN IF x.mode & Cond THEN loadCond(y) END; x.a := merge(y, x.b); x.b := y; x.r := x.y END And2; A negative consequence of this scheme having condition flags in the processor is that when an item with mode Cond has to be transferred into mode Reg, as in a Boolean assignment, an unpleasantly complex instruction sequence must be generated. Fortunately, this case occurs quite rarely. ## 12.7. Procedures Before embarking on an explanation of procedure calls, entries and exits, we need to know how recursion is handled and how storage for local variables is allocated. Procedure calls cause a sequence of frames to be allocated in a stack fashion. These frames are the storage space for local variables. Each frame is headed by a single word containing the return address of the call. This address is deposited in R15 by the call instructions (BL, branch and link). The compiler "knows" the size of the frame to be allocated, and thus merely decrements the stack pointer SP (Y8) by this amount. Upon return, SP is incremented by the same amount, and PC is restored by a branch instruction. In the following example, a procedure P is called, calling itself Q, and Q calling P again (recursion). The stack then contains 3 frames (see Figure 12.7). ``` #################### File: Project%20Oberon%20The%20Design%20of%20an%20Operating%20System%2C%20a%20Compiler%2C%20and%20a%20Computer%20-%20Niklaus%20Wirth%2C%20J%C3%BCrg%20Gutknecht%20%28PDF%29%20PO.Applications.pdf Page: 47 Context: # 12.7 Stack frames Scheme and layout determine the code sequences for call, entry, and exit of procedures. Here is an example of a procedure P with 2 parameters: **Call:** ``` LDR R0, param0 LDR R1, param1 BL P ``` **Prolog:** ``` SUB SP, SP, size // decrement SP STR LNK, SP, 0 // push return adr STR R0, SP, 4 // push parameter 0 STR R1, SP, 8 // push parameter 1 .... ``` **Epilog:** ``` LDR LNK, SP, 0 // pop return adr ADD SP, SP, size // increment SP BR LNK ``` When the call instruction is executed, parameters reside in registers, starting with R0. For function procedures, the result is passed in register R0. This scheme is very efficient; storing the parameters occurs only in a single place, namely at procedure entry, and not before each call. However, it has severe consequences for the entire register allocation strategy. Throughout the compiler, registers **must** be allocated in strict stack fashion. Furthermore, parameter allocation must start with R0. This is a distinct drawback for function calls. If registers are occupied by other values loaded prior to the call, they must be cleared, i.e., the parameters must be saved and reloaded after return. This is rather cumbersome (see procedures ORG.SaveRegisters and ORG.RestoreRegisters). | F(x) | no register saving | |--------------------|-------------------| | F(x) | | | (x+1) + F(x) | register saving necessary | ## 12.8 Type extension Static typing is an important principle in programming languages. It implies that every constant, variable, or function is of a certain data type, and that this type can be derived by reading the program text without executing it. It is the key principle to introduce important redundancy in languages in such a form that a compiler can detect inconsistencies. It is therefore the key element for reducing the number of errors in programs. However, it also acts as a restriction. It is, for example, impossible to construct data structures (arrays, trees) with different types of elements. In order to relax the rule of strictly static typing, the notion of **type extension** was introduced in Oberon. It makes it possible to construct. #################### File: Project%20Oberon%20The%20Design%20of%20an%20Operating%20System%2C%20a%20Compiler%2C%20and%20a%20Computer%20-%20Niklaus%20Wirth%2C%20J%C3%BCrg%20Gutknecht%20%28PDF%29%20PO.Applications.pdf Page: 48 Context: TYPE R0 = RECORD u: INTEGER END; R1 = RECORD (R0) u: INTEGER END; We say that R1 is an extension of R0. R0 has the fields u and v, R1 has u, v, and w. The concept becomes useful in combination with pointers. Let ``` TYPE P = POINTER TO R0; P1 = POINTER TO R1; VAR p0: P0; p1: P1; ``` Now it is possible to assign p1 to p0 (because a P1 is always also a P0), but not p0 to p1, because a P0 need not be a P1. This has the simple consequence that a variable of type P0 may well point to an extension of R0. Therefore, data structures can be declared with a base type, P0, as common element type, but in fact they can individually differ; they can be any extension of the base type. Obviously, it must be possible to determine the actual, current type of an element even if the base type is statically fixed. This is possible through a type test, syntactically a Boolean factor: ``` p0 IS P1 ``` (short for `p0 IS R1`) Furthermore, we introduce the type guard. In the present example, the designator `p0.v` is illegal, because there is no field `v` in a record of type P0, even if the current value of p0 is a R1. As this case occurs frequently, we introduce the short notation `p0(P1,p)`, implying a test `p0 IS P1` and an abort if the test is not met. It is important to mention that this technique also applies to formal variable parameters of record type, as they also represent a pointer to the actual parameter. Its type may be any extension of the type specified for the formal parameter in the procedure heading. How are type test and type guard efficiently implemented? Our first observation is that they must consist of a single comparison only, similar to index checks. This in turn implies that types must be identified by a single word. The solution lies in using the unique address of the type descriptor of the (record) type. Which data must this descriptor hold? Essentially, type descriptors (TD) must identify the base types of a given type. Consider the following hierarchy: ``` TYPE T = RECORD ... END; T0 = RECORD (T) ... END; // extension level 1 T1 = RECORD (T) ... END; // extension level 1 T00 = RECORD (T0) ... END; // extension level 2 T10 = RECORD (T1) ... END; // extension level 2 T11 = RECORD (T1) ... END; // extension level 2 ``` #################### File: Project%20Oberon%20The%20Design%20of%20an%20Operating%20System%2C%20a%20Compiler%2C%20and%20a%20Computer%20-%20Niklaus%20Wirth%2C%20J%C3%BCrg%20Gutknecht%20%28PDF%29%20PO.Applications.pdf Page: 49 Context: # Type Hierarchy In the symbol table, the field base refers to the ancestor of a given record type. Thus base of the type representing T11 points to T1, etc. Run-time checks, however, must be last, and hence cannot proceed through chains of pointers. Instead, each TD contains an array with references to the ancestor TDs (including itself). For the example above, the TDs are as follows: - `TD(T)` = `[]` - `TD(TI)` = `[T, TI]` - `TD(T00)` = `[T, T00]` - `TD(T01)` = `[T, T01]` - `TD(T10)` = `[T, T10]` - `TD(T11)` = `[T, T11]` Evidently, the first element can be omitted, as it always refers to the common base of the type hierarchy. The last element always points to the TD's owner. TDs are allocated in the data area, the area for variables. ## References to TDs References to TDs are called type tags. They are required in two cases. The first is for records referenced by pointers. Such dynamically allocated records carry an additional, hidden field holding their type tag. (A second additional word is reserved for use by the garbage collector. The offset of the tag field is therefore -8.) The second case is that of record-typed VAR-parameters. In this case, the type tag is explicitly passed along with the address of the actual parameter. Such parameters therefore require two words/registers. A type test consists of a test for equality of two type tags. In PS, the first tag is that of the n-th entry of the TD of `p`, where `n` is the extension level of `T`. The second tag is that of type `T`. This is shown in Pattern 1 in Section 12.2 (see also Fig. 12.4). The test then is as follows: - `p.tag[tl] = addr(T)`, where `l` is the extension level of `T`. When declaring a record type, it is not known how many extensions, or how many levels will be built on this type. Therefore, TDs should actually be infinite arrays. We decided to restrict them to 3 levels only. The first entry, which is never used for checking, is replaced by the size of the record. ## 12.7.9 Import and Export, Global Variables Addresses of imported objects are not available to the compiler. Their computation must be left to the module loader (see Chapter 6). Similar to handling addresses of imported modules, the compiler puts the necessary information in place of the actual addresses into the instruction itself. In the case of procedure calls, this is quite feasible, because the BL instruction features an offset of 24 bits. The information consists of the module number and the export number of the imported object. In addition, there is a link to the previous instruction referring to an imported procedure. The origin of the first imported circular jump is rooted in the compiler variable `fork`, and the 24 bits in each BL instruction 4 bits are used for the module number, 8 bits for the object's export number, and 12 for the link. The loader need only scan this list to fix up the addresses (jump offsets). Matters are more complex in the case of data. Object records in the symbol table have a field lev. It indicates the nesting level of variables local to procedures. It is also used for the module number in the case of variables of imported modules. Note that when importing, objects designating modules are inserted in the symbol table, and the list of their own objects are attached in the field `disc`. In this #################### File: Project%20Oberon%20The%20Design%20of%20an%20Operating%20System%2C%20a%20Compiler%2C%20and%20a%20Computer%20-%20Niklaus%20Wirth%2C%20J%C3%BCrg%20Gutknecht%20%28PDF%29%20PO.Applications.pdf Page: 50 Context: # 12.7.10 Traps This compiler provides an extensive system of safeguards by providing run-time checks (aborts) in several cases: | trap number | trap cause | |-------------|----------------------------------| | 1 | array index out of range | | 2 | type guard failure | | 3 | array or string copy overflow | | 4 | access via NIL pointer | | 5 | illegal procedure call | | 6 | integer division by zero | | 7 | assertion violated | These checks are implemented very efficiently in order not to downgrade a program's performance. Involved is typically a simple compare instruction, plus a conditional branch (BLR MT). It is assumed that any entry of the module table contains not a base address (module numbers start with 1), but a branch instruction to an appropriate trap routine. The trap number is encoded in bits 47:0 of the branch instruction. The predefined procedure `Assert` generates a conditional trap with trap number 7. For example, the statement `Assert(m = n)` generates: ``` LDR R0, m LDR R1, n CMP R0, R1 BLR R1, 7CH ; branch and link if unequal through R12 (MT), trap number 7 ``` Procedure `New`, representing the operator `NEW`, has been implemented with the aid of the trap mechanism. (This is in order to omit any reference to module `Kernel`, which contains the allocation procedure `New`). The generated code for the statement `NEW(p)` is: ``` ADD R0, R0, SP ; address of p ADD R1, SB, tag ; type tag BLR R1, 7CH ; branch and link unconditionally through R12 (MT), trap number 0 ``` #################### File: Project%20Oberon%20The%20Design%20of%20an%20Operating%20System%2C%20a%20Compiler%2C%20and%20a%20Computer%20-%20Niklaus%20Wirth%2C%20J%C3%BCrg%20Gutknecht%20%28PDF%29%20PO.Applications.pdf Page: 51 Context: # 13 A graphics editor ## 13.1 History and goal The origin of graphics systems as they are in use at this time was intimately tied to the advent of the high-resolution bit-mapped display and of the mouse as pointing device. The author's first contact with such equipment dates back to 1976. The Alto computer at the Xerox Palo Alto Research Center is justly termed the first workstation featuring those characteristics. The designer of its first graphics package was Ch. Thacker who perceived the usefulness of the high-resolution screen for drawing and processing schematics of electronic circuits. This system was cleverly tailored to the needs encountered in this activity, and it was remarkable in its compactness and effectiveness due to the lack of unnecessary facilities. Indeed, its acronym was SIL, for Simple Illustrator. After careful study of the used techniques, the author designed a variant, programmed in Modula-2 (instead of BCPL) for the PDP-11 Computer, thereby ordering and exhibiting the involved data structures more explicitly. In intervals of about two years, that system was revised and grew gradually into the present Draw system. The general goal remained a simple line drawing system: emphasis was placed on a clear structure and increase of flexibility through generalization of existing rather than indiscriminate addition of new features. In the history of this evolution, three major transitions can be observed. The first was the move from a single “window”, the screen, to multiple windows showing different excerpts of the same graphic. This step was performed on the Lilith computer which resembled the Alto in many ways. The second major transition was the application of the object-oriented style of programming, which allowed the addition of new element types to the basic system, making it extensible. The third step concerned the proper integration of the Draw system with Oberon’s text system. The last two steps were performed using Oberon and the Ceres computer. We refrain from exhibiting this evolution and merely present the outcome, although the history might be an interesting reflection of the evolution of programming techniques in general, containing many useful lessons. We stress the fact, however, that the present system rests on a long history of development, during which many features and techniques were introduced and later discarded or revised. The size of the system's description is a poor measure of the effort that went into its construction; deletion of program text sometimes marks bigger progress than addition. The goal of the original SIL program was to support the design of electronic circuit diagrams. Primarily, SIL was a line drawing system. This implies that the drawings remain uninterrupted. However, in a properly integrated system, the addition of modules containing operators that interpret the drawings is a reasonably straightforward proposition. In fact, the Oberon system is ideally suited for such steps, particularly due to its command facility. At first, we shall ignore features specifically tailored to circuit design. The primary one is a macro facility to be discussed in a later chapter. The basic system consists of the modules Draw, GraphicFrames, and Graphics. These modules contain the facilities to generate and handle horizontal and vertical lines, text captions, and macros. Additional modules serve to introduce other elements, such as rectangles and circles, and the system is extensible. Further modules may be introduced to handle further types of elements. ## 13.2 A brief guide to Oberon's line drawing system In order to provide the necessary background for the subsequent description of the Draw system's implementation, a brief overview is provided in the style of a user's guide. It summarizes the facilities offered by the system and gives an impression of its versatility. The system called Draw serves to prepare line drawings. They contain lines, text captions, and other items, and are displayed in graphic viewers (more precisely: in menu viewers' graphic frames). #################### File: Project%20Oberon%20The%20Design%20of%20an%20Operating%20System%2C%20a%20Compiler%2C%20and%20a%20Computer%20-%20Niklaus%20Wirth%2C%20J%C3%BCrg%20Gutknecht%20%28PDF%29%20PO.Applications.pdf Page: 52 Context: # Graphic Viewer Commands The graphic viewer shows an excerpt of the drawing plane, and several viewers may show different parts of a drawing. The most frequently used commands are built-in as mouse clicks and combinations of clicks. Additional commands are selectable from texts, either in viewer menus (title bars) or in the text called **Draw Tool**. Fig. 13.1 shows the display with two graphic viewers at the left and the draw tool text at the right. The mouse buttons have the following principal functions whenever the cursor lies in a graphic frame: - **left:** draw / set caret - **middle:** move / copy - **right:** select ## 13.2.1 Basic Commands The command **Draw.Open** opens a new viewer and displays the graph with the name given as a parameter. We suggest that file names use the extension `.graph`. ### Drawing a Line In order to draw a horizontal or vertical line from P0 to P1, the left key is pressed with the cursor at P0 and, while the key is held, the mouse and cursor is moved to P1. Then the key is released. If P0 and P1 differ in both their x and y coordinates, then the end point is adjusted so that the line is either horizontal or vertical. ### Writing a Caption First, the cursor is positioned where the caption is to appear. Then the left key is clicked, causing a crosshair to appear. It is called the caret. Then the text is typed. Only single lines of text are accepted. The DEL key may be used to retract characters (backspace). ### Selecting Elements Most commands require the specification of operands, and many implicitly assume the previously selected elements - the **selection** - to be their operands. A single element is selected by pointing at it with the cursor and then clicking the right mouse button. This also causes previously selected elements to be deselected. If the left key is also clicked, their selection is retained. This action is called an **intertick**. To select several elements at once, the cursor is moved from P0 to P1 while the right key is held. Then all elements lying within the rectangle with diagonally opposite corners at P0 and P1 are selected. Selected lines are displayed as dotted lines, selected captions (and macros) by inverse video mode. A macro is selected by pointing at its lower left corner. The command is called **sensitive area**. ### Moving Elements To move (displace) a set of elements, the elements are first selected and then the cursor is moved from P0 to P1 while the middle key is held. The vector from P0 to P1 specifies the movement and is called the **displacement vector**. P0 and P1 may lie in different viewers displaying the same graph. Small displacements may be achieved by using the keyboard's cursor keys. ### Copying Elements Similarly, the selected elements may be copied (duplicated). In addition to pressing the middle key indicating the displacement vector, the left key is interlocked. The copy command may also be used to copy elements from one graph into another graph by moving the cursor from one viewer into another viewer displaying the destination graph. A text item may be copied from a text image into a graphic frame and vice-versa. There exist two ways to accomplish this: 1. First the caret is placed at the destination position, then the text is selected and the middle key is interlocked. 2. First the text is selected, then the caret is placed at the destination position and the middle key is interlocked. ### Shifting the Plane You may shift the entire drawing plane behind the viewer by specifying a displacement vector pressing the middle button (like in a move command) and interlocking the right button. | Action | Mouse Button | |----------------------|---------------| | left (draw line) | draw line | | left (no motion) | set caret | #################### File: Project%20Oberon%20The%20Design%20of%20an%20Operating%20System%2C%20a%20Compiler%2C%20and%20a%20Computer%20-%20Niklaus%20Wirth%2C%20J%C3%BCrg%20Gutknecht%20%28PDF%29%20PO.Applications.pdf Page: 53 Context: ``` left + middle copy selected caption to caret left + right set secondary caret middle + left copy selection middle + right shift drawing plane right select area right (no motion) select object right + middle copy caption to caret right + left select without deselection ## 13.2.2. Menu commands The following commands are displayed in the menu (title bar) of every graphic viewer. They are activated by being pointed at and by clicking the middle button. - **Draw.Delete** The selected elements are deleted. - **Draw.Store** The drawing is written as a file with the name shown in the title bar. - **Draw.Restore** The original file is renamed by appending “.Bak”. - **Draw.Ticks** The frame displays a pattern of dots (ticks) to facilitate positioning. The two viewers in Fig. 13.1 display different parts of the same graphic. The second view was obtained from the generic **System.Copy** command and a subsequent shift of the drawing plane. ![Figure 13.1](path/to/image) *Figure 13.1 Display with graphics duplicated viewers* ## 13.2.3. Further commands The following commands are listed in the text **Draw.Tool**, but may appear in any text. - **Draw.Store name** The drawing in the marked viewer is stored as a file with the specified name. ``` Image Analysis: ### Comprehensive Examination of Visual Content #### 1. Localization and Attribution 1. **Image 1:** Position - Top right quadrant of the document. 2. **Image 2:** Position - Bottom of the document, divided into two sections next to each other. 3. **Image 3:** Position - Second section in the bottom right. #### 2. Object Detection and Classification 1. **Image 1:** - **Objects:** Command list with mouse actions. - **Category:** Text and instructions. - **Key Features:** Detailed description of commands using different mouse button combinations on a graphical interface. 2. **Image 2 & 3:** - **Objects:** Screen display of a graphic viewer illustrating various parts of the same graphic. - **Category:** Interface screenshot. - **Key Features:** Display of rectangles, circles, command lists, and other UI elements arranged over four segmented viewer sections. #### 3. Scene and Activity Analysis 1. **Image 1:** - **Scene:** A list of commands and actions achievable by mouse interactions. - **Activities:** Explanation of functionalities such as 'copy selected caption to caret', 'move selection’, and 'select object'. 2. **Image 2 & 3:** - **Scene:** Interface screenshots showing graphic viewer displaying different parts of an identical graphic. - **Activities:** Representation of what users would see on their screens when interacting with graphic viewer commands, the switching, and redrawing of graphic elements. #### 4. Text Analysis 1. **Extracted Text:** - **Top Text:** Describes mouse commands to manipulate graphics. - Example: "left + middle copy selected caption to caret". - **Middle Text:** Menu commands in a graphics viewer. - Example: "Draw.Delete The selected elements are deleted". - **Bottom Text:** Explanation of figure 13.1. - Example: "The two viewers in Fig. 13.1. display different parts of the same graphic." 2. **Content Analysis:** - The text provides detailed instructions and descriptions for using specific commands within a graphic interface. Each command and its action is clearly documented for user comprehension and application in graphic manipulation. #### 9. Perspective and Composition - **Perspective:** - The perspective is a direct, eye-level view commonly used in instructional documents to ensure clarity. - **Composition:** - The composition is methodical, with text sections followed by illustrative aids (images) providing context and examples to the textual instructions. #### 10. Contextual Significance - **Overall Document:** - This page is part of an instructional document on graphics viewers. - **Image Contribution:** - Each image enhances comprehension by visually representing commands and their effects, aiding users in understanding the interface's functionalities. #### 13. Graph Numbers No graphs with numerical lists were present, making this aspect not applicable. ### Additional Aspects Included - **Prozessbeschreibungen (Process Descriptions):** - **Process:** Instructions and commands for manipulating graphics. - **Description:** Step-by-step detailing of graphical command inputs and outputs using the graphical viewer interface. ### Summary: The document is an instructional guide on using a graphics viewer interface, focusing on mouse commands and their associated functions. The images clarify the text by showcasing the practical application within the interface. The commands range from basic selections to more complex tasks like restoring frames and saving drawings. The composition and contextual relevance of the images significantly aid in user comprehension and practical application of these commands. #################### File: Project%20Oberon%20The%20Design%20of%20an%20Operating%20System%2C%20a%20Compiler%2C%20and%20a%20Computer%20-%20Niklaus%20Wirth%2C%20J%C3%BCrg%20Gutknecht%20%28PDF%29%20PO.Applications.pdf Page: 54 Context: The subsequent commands change attributes of drawing elements, such as line width, text font, and color, and they apply to the most recent selection. ```markdown Draw.SetWidth w default = 1, 0 < w < 7 Draw.ChangeFont fontname Draw.ChangeColor c Draw.ChangeWidth w (0 < w < 7) The ChangeColor command either takes a color number in the range 1..15 or a string as a parameter. It serves to copy the color from the selected character. ## 13.2.4. Macros A macro is a (small) drawing that can be identified as a whole and be used as an element within a (larger) drawing. Macros are typically stored in collections called libraries, from where they can be selected and copied individually. `Draw.Macro lib mac` The macro `mac` is selected from the library named `lib` and inserted in the drawing at the caret's position. An example for the use of macros is drawing electronic circuit diagrams. The basic library file containing frequently used TTL components is called `TTLO.Lib`, and a drawing showing its elements is called `TTLO.Graph` (see Figure 13.2). ![Figure 13.2 Viewer with circuit macros of TTL0 library](path_to_image) ## 13.2.5. Rectangles Rectangles can be created as individual elements. They are frequently used for framing sets of elements. Rectangles consist of four lines that are selectable as a unit. The attribute commands `Draw.SetWidth`, `System.SetColor`, `Draw.ChangeWidth`, and `Draw.ChangeColor` also apply to rectangles. ``` Image Analysis: ### Analysis of the Visual Content #### 1. **Localization and Attribution:** - **Image 1**: - Located prominently below the text under the section "13.2.4 Macros". - Contains diagrams and text. - Numbered as "Image 1." #### 2. **Object Detection and Classification:** - **Image 1**: - Objects Detected: - Circuit diagrams with several components. - Text boxes and interface elements related to circuit design. - Classification: - Electronic components (logic gates, connectors). - Software interface windows. #### 3. **Scene and Activity Analysis:** - **Image 1:** - Entire Scene: - Depicts a software interface displaying circuit diagrams and components. - Activities: - Designing and viewing electronic circuit diagrams. - Main Actors: - The interface itself and its components like logic gates and connectors are the central focus, representing the activity of macro drawing and circuit design. #### 4. **Text Analysis:** - **Image 1:** - Extracted Text: - Various text snippets such as "Add Macro", "Edit Color", circuit names, pin assignments, etc. - Significance: - The text elaborates on the functions and operations available within the software, aiding users in designing and manipulating the electronic circuits. #### 5. **Diagram and Chart Analysis:** - **Image 1:** - Diagrams: - Electronic circuit diagrams showing connections and logical flow. - Some diagrams present logical gates and their interconnections. - Key Insights: - The diagrams represent simple to moderately complex electronic configurations utilizing TTL components. #### **Ablaufprozesse (Process Flows):** - **Image 1:** - Describes the process of designing electronic circuits using schematic diagrams. - Highlights the use of macros in simplifying complex designs by using predefined components. #### **Prozessbeschreibungen (Process Descriptions):** - The text and diagrams together describe the process of drawing and editing circuit designs using TTL components. Instructions include macro selection and insertion, changing attributes of drawing elements, and utilizing libraries. #### **Typen Bezeichnung (Type Designations):** - **TTL (Transistor-Transistor Logic) Components**: - Specifically mentions TTL components and associated tools like `TTL.Lib` for library and `TTL0.Graph` for example drawing. #### **Trend and Interpretation:** - The textual and visual content together suggest a trend towards using macros to streamline electronics design processes, focusing on reusability and efficiency in creating complex circuits. #### **Contextual Significance:** - **Image 1:** - It offers a visual representation to support the textual explanation regarding the usage of macros in circuit design. - Enhances understanding by providing a practical example (`Figure 13.2`). ### Conclusion The analyzed visual content primarily pertains to electronic circuit design, illustrating the use of macros, attributes modification, and libraries for efficient schematic creation. The diagrams and text together serve an educational purpose, explaining how to utilize features of a specific software for circuit design. #################### File: Project%20Oberon%20The%20Design%20of%20an%20Operating%20System%2C%20a%20Compiler%2C%20and%20a%20Computer%20-%20Niklaus%20Wirth%2C%20J%C3%BCrg%20Gutknecht%20%28PDF%29%20PO.Applications.pdf Page: 55 Context: # 13.2.6. Oblique lines, circles, and ellipses Further graphic elements are (oblique) lines, circles, and ellipses. The sensitive area of circles and ellipses is at their lowest point. They are created by the following steps: ## Lines: 1. The caret is placed where the starting point is to lie. 2. A secondary caret is placed at the position of the end. 3. The command `Curves.MakeLine` is activated. ## Circles: 1. The caret is placed where the circle’s center is to lie. 2. A secondary caret is placed, its distance from the center specifying the radius. 3. The command `Curves.MakeCircle` is activated. ## Ellipses: 1. The caret is placed where the center is to lie. 2. A second caret is placed. Its horizontal distance from the first caret specifies one axis. 3. A third caret is placed. Its vertical distance from the first caret specifies the other axis. 4. The command `Curves.MakeEllipse` is activated. # 13.2.7. Spline curves Spline curves are created by the following steps: 1. The caret is placed where the starting point is to lie. 2. Secondary carets are placed at the spline's fixed points (at most 20). 3. The command `Splines.MakeOpen` or `Splines.MakeClosed` is activated. # 13.2.8. Constructing new macros A new macro is constructed and inserted in the library `lib` under the name `mac` as follows: 1. All elements which belong to the new macro are selected. 2. The caret is placed at the lower left corner of the area to be spanned by the macro. 3. A secondary caret is placed at the upper right corner of the area to be spanned. 4. The command `MacroTool.MakeMacro lib` is activated. An existing macro can be decomposed (opened) into its parts as follows: 1. The macro is selected. 2. The caret is placed at the position where the decomposition is to appear. 3. The command `MacroTool.OpenMacro` is activated. # 13.3. The core and its structure Like a text, a graphic consists of elements, subsequently to be called objects. Unlike a text, which is a sequence of elements, a graphic is an unordered set of objects. In a text, the position of an element need not be explicitly indicated (stored); it is recomputed from the position of its predecessor each time it is needed, for example for displaying or selecting an element. In a graphic, each object must carry its position explicitly, as it is independent of any other object in the set. This is an essential difference, requiring a different treatment and much more storage space for an equal number of objects. #################### File: Project%20Oberon%20The%20Design%20of%20an%20Operating%20System%2C%20a%20Compiler%2C%20and%20a%20Computer%20-%20Niklaus%20Wirth%2C%20J%C3%BCrg%20Gutknecht%20%28PDF%29%20PO.Applications.pdf Page: 57 Context: ``` PROCEDURE drawObj(obj: Object); BEGIN IF obj IS Line THEN drawLine(obj(Line)) ELSE IF obj IS Caption THEN drawCaption(obj(Caption)) ELSE (* other object types, if any *) END; END drawObj; PROCEDURE drawGraphic(first: Object); VAR obj: Object; BEGIN obj := first; WHILE obj <> NIL DO drawObj(obj); obj := obj.next END; END drawGraphic; The two procedures typically are placed in different modules, one containing operations on objects, the other those on graphics. Here the former is the service module, the latter the former's client. Procedures for, e.g., copying elements, or determining whether an object is selectable, follow the same pattern as drawGraphic. This solution has the unpleasant property that all object types are anchored in the base module. If any new types are to be added, the base module has to be modified (and all clients are to be - at least - recompiled). The object-oriented paradigm eliminates this difficulty by inverting the roles of the two modules. It rests on binding the operations pertaining to an object type to each object individually in the form of procedure-typed record fields as shown in the following sample declaration: ObjectDesc = RECORD x, y, w, h: INTEGER; selected: BOOLEAN; draw: PROCEDURE (obj: Object); write: PROCEDURE (obj: Object; VAR f: Files.Rider); next: ObjectDesc END; The procedure drawGraphic is now formulated as follows: PROCEDURE drawGraphic(first: Object); VAR obj: Object; BEGIN obj := first; WHILE obj <> NIL DO drawObj(obj); obj := obj.next END; END drawGraphic; The individual procedures - in object-oriented terminology called methods - are assigned to the record's fields upon its creation. They need no further discrimination of types, as this role is assumed by the assignment of the procedures upon their installation. We note here that the procedure fields are never changed; they assume the role of constants rather than variables associated with each object. This example exhibits in a nutshell the essence of object-oriented programming, extensibility as its purpose and the procedure-typed record field as the technique. The given solution, as it stands, has the drawback that each object (instance, variable) contains several procedures (of which three are listed), and therefore leads to a storage requirement that should be avoided. Furthermore, it defines once and for all the number of operations applicable to objects, and also their parameters and result types. A different approach with the same underlying principle removes these drawbacks. It employs a single installed procedure which itself discriminates among the operations according to differing types of parameters. The parameters of the preceding solution are merged into a single record called a message. The unified procedure is called a handler, and messages are typically extensions of a single base type Msg. TYPE Msg = RECORD END; DrawMsg = RECORD (Msg) END; WriteMsg = RECORD (Msg: Files.Rider END; ObjectDesc = RECORD x, y, w, col: INTEGER; selected: BOOLEAN; END; ``` #################### File: Project%20Oberon%20The%20Design%20of%20an%20Operating%20System%2C%20a%20Compiler%2C%20and%20a%20Computer%20-%20Niklaus%20Wirth%2C%20J%C3%BCrg%20Gutknecht%20%28PDF%29%20PO.Applications.pdf Page: 58 Context: # Modular Structure of Object Management ## Procedures ### handle ```pascal PROCEDURE handle (obj: Object; VAR M: Msg); next: Object END; ``` ### Handler ```pascal PROCEDURE Handler (obj: Object; VAR M: Msg); BEGIN (* this procedure is assigned to the handle field of every line object *) IF M IS DrawMsg THEN drawLine(obj, Line) ELSIF M IS WriteMsg THEN writeLine(obj, M(WriteMsg).R) ELSE ... END; END; ``` ### drawGraphic ```pascal PROCEDURE drawGraphic(first: Object; VAR M: Msg); VAR obj: Object; BEGIN obj := first; WHILE obj <> NIL DO obj.handle(obj, M); obj := obj.next END END drawGraphics; ``` ## Method Record Structure In the present system, a combination of the two schemes presented so far is used. It eliminates the need for individual method fields in each object record as well as the cascaded IF statement for discriminating among the message types. Yet it allows further addition of new methods for later extensions without the need to change the object's declaration. The technique used is to include a single field (called `obj`) in each record (analogous to the handler). This field is a pointer to a method record containing the procedures declared for the base type. At least one of them uses a message parameter, i.e., a parameter of record structure that is extensible. ### Type Definitions ```pascal TYPE Method = POINTER TO MethodDesc; Msg = RECORD END; Context = RECORD END; Object = POINTER TO ObjectDesc; ObjectDesc = RECORD x, y, w, h: INTEGER; selected: BOOLEAN; method: POINTER TO Method; next: Object; END; MethodDesc = RECORD new: Modules.Command; copy: PROCEDURE (obj: Object; to: Object); draw: handle: PROCEDURE (obj: Object; VAR M: Msg); selectable: PROCEDURE (obj: Object; x, y: INTEGER): BOOLEAN; read: PROCEDURE (obj: Object; VAR R: Files.Rider; VAR C: Context); write: PROCEDURE (obj: Object; con: INTEGER; VAR R: Files.Rider; VAR C: Context); END; ``` A single method instance is generated when a new object type is created, typically in the initialization sequence of the concerned module. When a new object is created, a pointer to its record is assigned to the do field of the new object descriptor. A call then has the form `obj.do.write(obj, R)`. This example exhibits the versatility of Oberon's type extension and procedure variable features very well, and it does so without hiding the data structures involved in a dispensable, built-in run-time mechanism. The foregoing deliberations suggest the system’s modular structure shown in Figure 13.3. #################### File: Project%20Oberon%20The%20Design%20of%20an%20Operating%20System%2C%20a%20Compiler%2C%20and%20a%20Computer%20-%20Niklaus%20Wirth%2C%20J%C3%BCrg%20Gutknecht%20%28PDF%29%20PO.Applications.pdf Page: 62 Context: # Figure 13.4 Frame and Graph Coordinates As a consequence, the display coordinates \( u, v \) of an object \( z \) of a graph displayed in a frame \( f \) are computed as: \[ u = z.x + t.x, \quad v = z.y + t.y \] In order to determine whether an object \( z \) lies within a frame \( f \), the following expression must hold: \[ (X.x \leq u) \& (u + z.w \leq f.X1) \& (Y.y \leq v) \& (v + z.h \leq f.Y1) \] The record field marked indicates whether or not the frame contains a caret. Its display position is recorded in the field called mark. A frame may contain several (secondary) carets; they form a list of individual location descriptors. When an object is displayed (drawn), its state must be taken into consideration in order to provide visible user feedback. The manner in which selection is indicated, however, may vary among different object types. This can easily be realized, because every object (type) is associated with an individual drawing procedure. The following visualizations of selection have been chosen: - Selected lines are shown in a gray tone (raster pattern). - Selected captions are shown with "inverse video". Change of state is a relatively frequent operation, and if possible a complete repainting of the involved objects should be avoided for reasons of efficiency. Therefore, procedures for drawing an object are given a mode parameter, in addition to the obvious object and frame parameters. The parameters are combined into the message record of type `DrawMsg`. ```pascal DrawMsg = RECORD (Graphics.Msg) f: Frame; mode: x, y, col: INTEGER; END; ``` The meaning of the mode parameter's four possible values are the following: - **mode = 0**: draw object according to its state, - **mode = 1**: draw reflecting a transition from normal to selected state, - **mode = 2**: draw reflecting a transition from selected to normal state, - **mode = 3**: erase. In the case of captions, for instance, the transitions are indicated by simply inverting the rectangular area covered by the caption. No rewriting of the captions' character patterns is required. #################### File: Project%20Oberon%20The%20Design%20of%20an%20Operating%20System%2C%20a%20Compiler%2C%20and%20a%20Computer%20-%20Niklaus%20Wirth%2C%20J%C3%BCrg%20Gutknecht%20%28PDF%29%20PO.Applications.pdf Page: 63 Context: A mode parameter is also necessary for reflecting object deletion. First, the selected objects are drawn with mode indicating erasure. Only afterwards are they removed from the graphic's linked list. Furthermore, the message parameter of the drawing procedure contains two offsets x and y. They are added to the object's coordinates, and their significance will become apparent in connection with macros. The same holds for the color parameter. The drawing procedures are fairly straightforward and use the four basic raster operations of module Display. The only complication arises from the need to clip the drawing at the frame boundaries. In the case of captions, a character is drawn only if it fits into the frame in its entirety. The raster operations do not test (again) whether the indicated position is valid. At this point we recall that copies of a viewer (and its frames) can be generated by the `System.Copy` command. Such copies display the same graphic, but possibly different excerpts of them. When a graphic is changed by an insertion, deletion, or any other operation, at a place that is visible in several frames, all affected views must reflect the change. A direct call to a drawing procedure indicating a frame and the change does therefore not suffice. Here again, the object-oriented style solves the problem neatly: In place of a direct call a message is broadcast to all frames, the message specifying the nature of the required updates. The broadcast is performed by the general procedure `Viewers.Broadcast(M)`. It invokes the handlers of all viewers with the parameter M. The viewer handlers either interpret the message or propagate it to the handlers of their subframes. Procedure `obj.handle` is called with a control message as parameter when pointing at the object and clicking the middle mouse button. This allows control to be passed to the handler of an individual object. The definition of module `GraphicFrames` is summarized by the following interface: ``` DEFINITION GraphicFrames; IMPORT Display, Graphics; TYPE Frame = POINTER TO FrameDesc; LocDesc = POINTER TO LocDesc; LocDesc = RECORD x, y: INTEGER; next: Location END; FrameDesc = RECORD (Display.FrameDesc) graph: Graphics.Graph; xg, Yx, Y1, y1: x, y: INTEGER; marked, ticked: BOOLEAN; mark: LocDesc END; (* mode = 0 - draw according to selected, 1: normal -> selected, 2: selected -> normal, 3: erase *) DrawMsg = RECORD (Graphics.Msg) t: Frame; x, y, col, mode: INTEGER END; PROCEDURE Restore (F: Frame); PROCEDURE Focus (F: Frame); PROCEDURE Selected (F: Frame); PROCEDURE This(x: INTEGER; F: Frame); PROCEDURE Draw (F: Frame); PROCEDURE Erase (F: Frame); PROCEDURE DrawObj (F: Frame; obj: Graphics.Object); PROCEDURE EraseObj (F: Frame; obj: Graphics.Object); PROCEDURE Change (F: Frame; VAR msg: Graphics.Msg); PROCEDURE Decease (F: Frame); PROCEDURE Deselect (F: Frame); PROCEDURE Macro (VAR Lname, Mname: ARRAY OF CHAR); ``` #################### File: Project%20Oberon%20The%20Design%20of%20an%20Operating%20System%2C%20a%20Compiler%2C%20and%20a%20Computer%20-%20Niklaus%20Wirth%2C%20J%C3%BCrg%20Gutknecht%20%28PDF%29%20PO.Applications.pdf Page: 65 Context: # 13.6. Macros For many applications, it is indispensable that certain sets of objects may be named and used as objects themselves. Such a named subgraph is called a **macro**. A macro thus closely mirrors the sequence of statements in a program text that is given a name and can be referenced from within other statements: the procedure. The notion of a graphic object becomes recursive, too. The facility of recursive objects is so fundamental that it was incorporated in the base module **Graphics** as the third class of objects. Its representation is straightforward: in addition to the attributes common to all objects, a field is provided storing the head of the list of elements which constitute the macro. In the present system, a special node is introduced representing the head of the element list. It is of type `MacHeadDesc` and carries also the name of the macro and the width and height of the rectangle covering all elements. These values serve to speed up the selection process, avoiding their recomputation by scanning the entire element list. The recursive nature of macros manifests itself in recursive calls of display procedures. In order to draw a macro, drawing procedures of the macro's element types are called (which may be macros again). The coordinates of the macro are added to the coordinates of each element, which function as offsets. The color value of the macro, also a field of the parameter of type `DrawSw`, overrides the colors of the elements. This implies that macros always appear monochrome. An application of the macro facility is the design of schematics of electronic circuits. Circuit components correspond to macros. Most components are represented by a rectangular frame and by labeled connectors (pins). Some of the most elementary components, such as gates, diodes, transistors, resistors, and capacitors are represented by standardized symbols. Such symbols, which may be regarded as forming an alphabet of electronic circuit diagrams, are appropriately provided in the form of a special font, i.e., a collection of raster patterns. Three such areas are shown in Figure 13.5, together with the components from which they are assembled. The definitions of the data types involved are: ``` Macro = POINTER TO MacroDesc; MacroDesc = RECORD (ObjectDesc) mac: MacHead END; MacHead = POINTER TO MacHeadDesc; MacHeadDesc = RECORD name: Name; w: INTEGER; lib: Library END; Library = POINTER TO LibraryDesc; LibraryDesc = RECORD name: Name END; ``` Procedure `DrawMac(mh, mh)` displays the macro with head `mh` according to the draw message parameter `mh` which specifies a frame, a position within the frame, a display mode, and an overriding color. In the majority of applications, macros are not created by their user, but are rather provided from another source, in the case of electronic circuits typically by the manufacturer of the components represented by the macros. As a consequence, macros are taken from a collection (inappropriately) called a **library**. In our system, a macro is picked from such a collection by the command `Draw.Macro` with a library name and a macro name as parameters. It inserts the specified macro at the place of the caret by calling `GraphicFrames.Macro`, which in turn calls `Graphics.Add`. At last, we mention that selection of a macro is visualized by covering with a dot pattern the entire rectangular area occupied by the macro. This emphasizes the fact that the macro constitutes an object as a whole. #################### File: Project%20Oberon%20The%20Design%20of%20an%20Operating%20System%2C%20a%20Compiler%2C%20and%20a%20Computer%20-%20Niklaus%20Wirth%2C%20J%C3%BCrg%20Gutknecht%20%28PDF%29%20PO.Applications.pdf Page: 67 Context: ```markdown # Modules.ThisCommand Then the data of the base type `Object` are read, and lastly the data of the extension are read by a call to the class method read. The following may serve as a template for any module defining a new object class `X`. Two examples are given in Section 13.9, namely **Rectangles** and **Curves**. ```oberon MODULE X; IMPORT Files, Oberon, Graphics, GraphicFrames; TYPE XType = POINTER TO XDesc; XDesc = RECORD (Graphics:ObjectDesc) (* additional data fields *) END; VAR method: Graphics.Method; PROCEDURE New*; VAR x: X; BEGIN NEW(x); x^.do := method; Graphics.new := x END New; PROCEDURE Copy(obj, to: Graphics.Object); BEGIN to^(x^) := obj(x^) END Copy; PROCEDURE Draw(obj: Graphics.Object; VAR msg: Graphics.Msg); BEGIN (* Draw *) END Draw; PROCEDURE Selectable(obj: Graphics.Object; x, y: INTEGER): BOOLEAN; BEGIN (* Selectable *) END Selectable; PROCEDURE Change(obj: Graphics.Object; VAR msg: Graphics.Msg); BEGIN IF msg IS Graphics.ColorMsg THEN obj^ := msg(Graphics.ColorMsg) ELSEIF msg IS THEN ... END END Change; PROCEDURE Read(obj: Graphics.Object; VAR w: Files.Rider; VAR C: Context); BEGIN READ(*read *) END Read; PROCEDURE Write(obj: Graphics.Object; cno: SHORTINT); VAR w: Files.Rider; VAR C: Context; BEGIN Graphics.WriteOBJ(w, con, obj); (* write *) END Write; PROCEDURE Make*; (* "command" *) VAR x: X; VAR gf: GraphicFrames.Frame; BEGIN NEW(x); x^.F.mark := X^.x; x^.F.mark := F.y; x^.w := ...; x^.col := Oberon.CurCol; x^.do := method; GraphicFrames.Defocus(F); Graphics.AddF(graf, x); GraphicFrames.DrawObj(F, x) END Make; BEGIN NEW(method); method.module := "X"; method.allocator := "New"; method.copy := Copy; method.draw := Draw; method.selectable := Selectable; method.handle := Handle; method.read := Read; method.write := Write; method.print := Print END X. ``` We wish to point out that also the macro and library facilities are capable of integrating objects of new classes, i.e., of types not occurring in the declarations of macro and library facilities. The complete interface definition of module `Graphics` is obtained from its excerpt given in Sect. 13.3, augmented by the declarations of types and procedures in Sect. 13.6 and 13.7. ``` #################### File: Project%20Oberon%20The%20Design%20of%20an%20Operating%20System%2C%20a%20Compiler%2C%20and%20a%20Computer%20-%20Niklaus%20Wirth%2C%20J%C3%BCrg%20Gutknecht%20%28PDF%29%20PO.Applications.pdf Page: 68 Context: # 13.8. The implementation ## 13.8.1. Module Draw Module `Draw` is a typical command module whose exported procedures are listed in a tool text. Its task is to scan the text containing the command for parameters, to check their validity, and to activate the corresponding procedures, which primarily are contained in modules `Graphics` and `GraphicFrames`. The most prominent among them is the `Open` command. It generates a new viewer containing two frames, namely a text frame serving as menu, and a graphic frame. We emphasize at this point that graphic frames may be opened and manipulated also by other modules apart from `Draw`. In particular, document editors that integrate texts and graphics - and perhaps also other entities - would refer to `Graphics` and `GraphicFrames` directly, but not make use of `Draw` which, as a tool module, should not have client modules. ``` DEFINITION Draw; PROCEDURE Open; PROCEDURE Delete; PROCEDURE SetWidth; PROCEDURE ChangeColor; PROCEDURE Store; PROCEDURE Macro; PROCEDURE OpenMacro; PROCEDURE MakeMacro; PROCEDURE LoadLibrary; END Draw. ``` ## 13.8.2. Module GraphicFrames Module `GraphicFrames` contains all routines concerned with displaying, visualizing graphic frames and their contents, i.e., graphics. It also contains the routines for creating new objects of the base classes, i.e., lines, captions, and macros. And most importantly, it specifies the appropriate frame handler which interprets input actions and thereby defines the user interface. The handler discriminates among the following message types: 1. **Update messages**. According to the id field of the message record, either a specific object or the entire selection of a graphic are drawn according to a mode. The case `id = 0` signifies a restoration of the entire frame including all objects of the graphic. 2. **Selection, focus, and position queries**. They serve for the identification of the graphic frame containing the latest selection, containing the caret (mark) or the indicated position. In order to identify the latest selection, the time is recorded in the graph descriptor whenever a new selection is made or when new objects are inserted. 3. **Input messages**. They originate from the central loop of module Oberon and indicate either a mouse action (track message) or a keyboard event (consume message). 4. **Control messages** from Oberon. They indicate that all marks (selection, caret, star) are to be removed (neutralize), or that the focus has to be relinquished (defocus). 5. **Selection and copy messages** from Oberon. They constitute the interface between the graphics and the text system, and make possible identification and copying of captions between graphic and text frames. 6. **Modify messages** from `Menu.Viewers`. They indicate that a frame has to be adjusted in size and position because a neighbouring viewer has been reshaped, or because its own viewer has been repositioned. 7. **Display messages**. They originate from procedure `InsertChar` and handle the displaying of single characters when a caption is composed (see below). #################### File: Project%20Oberon%20The%20Design%20of%20an%20Operating%20System%2C%20a%20Compiler%2C%20and%20a%20Computer%20-%20Niklaus%20Wirth%2C%20J%C3%BCrg%20Gutknecht%20%28PDF%29%20PO.Applications.pdf Page: 80 Context: The tool performing such a scan is called **Scavenger**. It is, like **DiskCheck**, a simple command interpreter with the following available commands: | parameters | action | |------------|--------------------------------------| | 0 s | send and mirror integer (test) | | 1 n | Scan the first n sectors and collect headers | | 2 - | Display names of collected files | | 3 - | Build new directory | | 4 - | Transfer new directory to the disk | | 5 - | Clear display | During the scan, a new directory is gradually built up in primary store. Sectors marked as headers are recorded by their name and creation date. The scavenger is the reason for recording the file name in the header, although it remains unused there by the Oberon System. Recovery of the date is essential because several files with the same name may be found. If one is found with a newer creation date, the older entry is overwritten. Command **W** transfers the new directory to the disk. For this purpose, it is necessary to have free sectors available. These have been collected during the scan: both old directory sectors (identified by a directory mark similar to the header mark) and overwritten headers are used as free locations. The scavenger has proven its worth on more than one occasion. Its main drawback is that it may rediscover files that had been deleted. The deletion operation by definition affects only the directory, but not the file. Therefore, the header carrying the name remains unchanged and is discovered by the scan. All in all, however, it is a small deficiency. ## Reference 1. N. Wirth. Designing a System from Scratch. *Structured Programming*, 1, (1989), 10-18. #################### File: Project%20Oberon%20The%20Design%20of%20an%20Operating%20System%2C%20a%20Compiler%2C%20and%20a%20Computer%20-%20Niklaus%20Wirth%2C%20J%C3%BCrg%20Gutknecht%20%28PDF%29%20PO.Applications.pdf Page: 81 Context: # 15 Tool and service modules In this chapter, a few modules are presented that do not belong to Oberon's system core. However, they belong to the system in the sense of being basic, and of assistance in some way, either to construct application programs, to communicate with external computers, or to analyze existing programs. ## 15.1 Basic mathematical functions Module `Math` contains the basic standard functions that had been postulated already in 1960 by Algol 60. They are: - `sqrt(x)` | the square root - `exp(x)` | the exponential function - `ln(x)` | the natural logarithm - `sin(x)` | the sine function - `cos(x)` | the cosine function They are presented here only briefly without discussing their approximation methods. However, we point out how advantage can be taken from knowledge about the internal representation of floating-point numbers. ### 15.1.1 Conversion between integers and floating-point numbers The Oberon System adopts the standard format postulated by IEEE. Here we restrict it to the 32-bit variant. A floating-point number `x` consists of 3 parts: - `s` | the sign | 1 bit - `e` | the exponent | 8 bits - `m` | the mantissa | 23 bits Its value is defined as \[ x = (-1)^s \times 2^{e-127} \times (1.m) \] A number is in normalized form, if its mantissa satisfies \(1.0 \leq m < 2.0\). It is assumed that numbers are always normalized, and therefore the leading 1-bit is omitted. The exception is the singular value 0, which cannot be normalized. It must therefore be treated as a special case. It follows that integers and floating-point numbers are represented quite differently, and that conversion operations are necessary to transfer a number from one format to the other. This is the reason why the Oberon language keeps the two types INTEGER and REAL separate. Conversion must be explicitly specified by using the two predefined functions: - \( n = \text{FLOOR}(x) \quad \text{REAL} \rightarrow \text{INTEGER} \) - \( x = \text{FLT}(n) \quad \text{INTEGER} \rightarrow \text{REAL} \) Note: `FLOOR(x)` rounds toward -inf. For example `FLOOR(1.5) = 1`, `FLOOR(-1.5) = -2`. The RISC processor does not feature specific instructions implementing these functions. Instead, the compiler generates inline code using the FAD instruction with special options suppressing normalization. This option is specified by the `u` and `v` modifier bits of the instruction. The `FLOOR` function is realized by adding 0 with an exponent of 127 + 24 and suppressing the insertion of a leading 1-bit (i.e. `u = 1`). This causes the mantissa of the argument to be shifted right until its exponent is equal to 151. The RISC instructions are: ``` MOV R1 R0 4800H R1 = 48000000H FAD R0 R0 R1 ``` The `FLT` function is implemented also by adding 0 with an exponent of 151 and forced insertion of a leading 1-bit (i.e. `v = 1`). #################### File: Project%20Oberon%20The%20Design%20of%20an%20Operating%20System%2C%20a%20Compiler%2C%20and%20a%20Computer%20-%20Niklaus%20Wirth%2C%20J%C3%BCrg%20Gutknecht%20%28PDF%29%20PO.Applications.pdf Page: 84 Context: ```markdown n := FLOOR(y + 0.5); y := (y - n) and then distinguish between two approximating polynomials depending on whether x < π/4. ```pascal PROCEDURE sink(VAR y: REAL; REAL); CONST p0 = 3.86917671E-1; (* (2/pi) *) p1 = 7.85395616E-2; p2 = 2.39904032E-3; p3 = 3.15796226E-5; p4 = 1.336126E-7; p5 = 1.75719436E-9; p6 = 6.67711094E-12; q0 = 9.999999E-1; q1 = 4.848149E-1; q2 = 1.583434E-2; q3 = 3.2599149E-4; q4 = 3.859915E-6; q5 = 2.4603875E-8; q6 = 1.136138E-10; VAR r: INTEGER; yv: REAL; BEGIN r := y < 0; IF y < 0 THEN n := FLOOR(y + 0.5) ELSE n := FLOOR(y - 0.5); y := y - 2.0 * r * y; IF (ord(y) < 0) THEN y := (((((p0*y + p1)*y + p2)*y + p3)*y + p4)*y + p5)*y + p6) / (((((q0*y + q1)*y + q2)*y + q3)*y + q4)*y + q5)*y + q6) ELSE y := (((((p5*y + p4)*y + p3)*y + p2)*y + p1)*y + p0) / (((((q6*y + q5)*y + q4)*y + q3)*y + q2)*y + q1)*y + q0)); IF ODODIN(DIV y) THEN END; RETURN; END; ``` 15.2 A data link -------------- Module **PCLink** serves to transfer data (files) to and from another system. Data are transmitted as a sequence of blocks. Each block is a sequence of bytes. The number of data bytes lies between 0 and 255. They are preceded by a single byte indicating the length. Blocks are 255 bytes long, except the last block, whose length is less than 255. Here, the transmission channel is an RS-232 line. The interface consists of two registers, one for a data byte (address = -56), and one for the status (address = -52). Bit 0 of this status register indicates whether a byte had been received. Bit 1 of the status register indicates, whether the byte in the data register has been sent. (Note: the default transmission rate of the RISC is 9600 b/s). This module represents a server running as an Oberon task which must be activated by the command run. A server running on the partner system must be the master issuing requests. The command sequence is a REC byte, a SND byte, or a REQ byte for letting the connection. REC and SND must be followed by a file name, and the sequence of blocks. Every block is acknowledged by the receiver sending an ACK byte, for which the sender waits before sending the next block. There is no synchronization between blocks. Because the writing into a file may involve operations of unpredictable duration, the received bytes are not written to the file immediately. They are buffered and only output after the entire block has been received. ```pascal MODULE PCLINK; (* NW 8.2.2018 for Oberon on RISC *) IMPORT SYSTEM, Files, Texts, Oberon; CONST data = -56; (* -52 *) blink = 255; VAR REQ = 20H; (* 21H = 21H; SND = 22H; ACK = 10H; NAK = 11H *) VAR O: Oberon.Task; X: TextWriter; PROCEDURE Rec(VAR x: BYTE); BEGIN ``` #################### File: Project%20Oberon%20The%20Design%20of%20an%20Operating%20System%2C%20a%20Compiler%2C%20and%20a%20Computer%20-%20Niklaus%20Wirth%2C%20J%C3%BCrg%20Gutknecht%20%28PDF%29%20PO.Applications.pdf Page: 85 Context: ``` REPEAT UNTIL SYSTEM.BIT(stat, 0); SYSTEM.GET(data, x); END Dec; PROCEDURE RecName(VAR s: ARRAY OF CHAR); VAR i: INTEGER; x: BYTE; BEGIN i := 0; RecX(i); WHILE x <> 0 DO BEGIN s[i] := CHR(x); INC(i); RecX; END; s[i] := 0; END RecName; PROCEDURE Send; BYTE; BEGIN REPEAT UNTIL SYSTEM.BIT(stat, 1); SYSTEM.PUT(data, x); END Send; PROCEDURE Task; VAR len, n: INTEGER; x, ack, code: BYTE; name: ARRAY[0..31] OF CHAR; F: Files.File; Ri: Files.Rider; buf: ARRAY[0..255] OF BYTE; BEGIN IF SYSTEM.BIT(stat, 0) THEN (* "byte available" *) Rec(code); code := SND THEN (* "send file" *) RecName(name); F := Files.Old(name); IF F = NIL THEN Send(ACK); len := Files.Length(F); Files.SetR(F, 0); REPEAT IF len > 1k THEN len := 1k; ELSE len := len; Send(len); n := len; len := len - 1; WHILE n > 0 DO Files.Read(buf, x); Send(buf); DEC(n); END UNTIL len < 1k; ELSE Send(1); END END; ELSE code := REC THEN (* "receive file" *) RecName(name); IF F = NIL THEN Files.SetR(F, 0); Send(ACK); REPEAT Rec(len); len := x - 1; WHILE len > 0 DO buf[i] := 0; INC(i); END; WHILE i < len DO Files.WriteByte(F, buf[i]); INC(i); END UNTIL len < 255; Files.Register(F); Send(ACK); ELSE Send(NAK); END END; ELSE code := REC THEN Send(ACK) (* "for testing" *) END; END; END Task; PROCEDURE Run; BEGIN Oberon.Inst(); Texts.WriteString(W, "PCLink started"); Texts.WriteLn(""); Texts.Append(Oberon.Log, W.buf); END Run; PROCEDURE Stop; ``` #################### File: BIOS%20Disassembly%20Ninjutsu%20Uncovered%201st%20Edition%20-%20Darmawan%20Salihun%20%28PDF%29%20POST_Memory_Manager_Specification.pdf Page: 1 Context: # POST Memory Manager Specification **Version 1.01** **November 21, 1997** This specification has been made available to the public. You are hereby granted the right to use, implement, reproduce, and distribute this specification with the foregoing rights at no charge. This specification is, and shall remain, the property of Phoenix Technologies Ltd. (“Phoenix”), and Intel Corporation (“Intel”). --- NEITHER PHOENIX NOR INTEL MAKE ANY REPRESENTATION OR WARRANTY REGARDING THIS SPECIFICATION OR ANY PRODUCT OR ITEM DEVELOPED BASED ON THIS SPECIFICATION. USE OF THIS SPECIFICATION FOR ANY PURPOSE IS AT THE RISK OF THE PERSON OR ENTITY USING IT. PHOENIX AND INTEL DISCLAIM ALL EXPRESS AND IMPLIED WARRANTIES, INCLUDING BUT NOT LIMITED TO THE IMPLIED WARRANTIES OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND FREEDOM FROM INFRINGEMENT. WITHOUT LIMITING THE GENERALITY OF THE FOREGOING, NEITHER PHOENIX NOR INTEL MAKE ANY WARRANTY OF ANY KIND THAT ANY ITEM DEVELOPED BASED ON THIS SPECIFICATION, OR ANY PORTION OF IT, WILL NOT INFRINGE ANY COPYRIGHT, PATENT, TRADE SECRET OR OTHER INTELLECTUAL PROPERTY RIGHT OF ANY PERSON OR ENTITY IN ANY COUNTRY. #################### File: BIOS%20Disassembly%20Ninjutsu%20Uncovered%201st%20Edition%20-%20Darmawan%20Salihun%20%28PDF%29%20POST_Memory_Manager_Specification.pdf Page: 2 Context: # TABLE OF CONTENTS 1. **INTRODUCTION**.............................................................................................3 1.1 **REVISION HISTORY**..................................................................................3 1.1.1 **Technical Editors**...............................................................................3 1.2 **RELATED DOCUMENTS**............................................................................3 1.3 **TERMS**....................................................................................................4 2. **FUNCTIONALITY**............................................................................................5 2.1 **OVERVIEW**..............................................................................................5 2.1.1 **Why a PMM?**...................................................................................5 2.1.2 **PMM Model**....................................................................................5 2.2 **CLIENTS**..................................................................................................5 3. **PMM SERVICES INTERFACE**.........................................................................6 3.1 **DETECTING PMM SERVICES**...................................................................6 3.1.1 **PMM Structure**................................................................................6 3.1.2 **Detection Algorithm**.....................................................................6 3.2 **PMM SERVICES**.....................................................................................6 3.2.1 **Interface Style**..............................................................................7 3.2.2 **Stack Requirements**.....................................................................7 3.2.3 **Memory Block Alignment**................................................................7 3.2.4 **Accessing Extended Memory**........................................................7 3.3 **PMMALLOCATE - FUNCTION 0**...............................................................7 3.3.1 **Description**....................................................................................7 3.3.2 **Function Prototype**........................................................................8 3.3.3 **Parameters**....................................................................................8 3.4 **PMMFREE - FUNCTION 1**......................................................................9 3.4.1 **Description**....................................................................................9 3.4.2 **Function Prototype**........................................................................9 3.4.3 **Parameters**....................................................................................9 3.5 **PMMDEALLOCATE - FUNCTION 2**..........................................................9 3.5.1 **Description**....................................................................................9 3.5.2 **Function Prototype**.......................................................................10 3.5.3 **Parameters**...................................................................................10 3.6 **C LANGUAGE CODING EXAMPLE**..........................................................11 3.7 **ASSEMBLY LANGUAGE CODING EXAMPLE**........................................11 4. **CONVENTIONS FOR CREATING MEMORY BLOCK HANDLES**......................14 4.1 **NAME SELECTION**.................................................................................14 4.1.1 **ISA Product Identifiers**.................................................................14 4.1.2 **Convention for Selecting Handle Values**....................................14 4.2 **RECOMMENDED METHOD FOR THE USE OF NAMED BLOCKS**...........15 4.3 **FINDING OTHER CARDS IN THE SYSTEM**...........................................16 #################### File: BIOS%20Disassembly%20Ninjutsu%20Uncovered%201st%20Edition%20-%20Darmawan%20Salihun%20%28PDF%29%20POST_Memory_Manager_Specification.pdf Page: 3 Context: # 1 Introduction ## 1.1 Revision History | Revision | Date | Changes | |----------|--------------------|---------------------------------------------------------------------------------------------| | 1.01 | November 21, 1997 | Included guidelines on using extended memory during POST. | | | | Clarified the processor mode and the state of Gate A20 during POST. | | | | Definitions for the terms: Big Real Mode, Extended Memory, and Gate A20 were added. | | | | Changed to not clear the contents of memory blocks when they are deallocated. | | | | Simplified the assembly language coding example by using 32-bit instructions and operands. | | | | Clarified the 'C' language code example by adding a function to find the PMM structure. | | 1.0 | September 20, 1996 | Approved for public release. | ## 1.1.1 Technical Editors **Scott Townsend** Phoenix Technologies Ltd. 135 Technology Drive Irvine, CA 92618 Phone: (714) 790-2125 Fax: (714) 790-2001 Email: [Scott.Townsend@Phoenix.com](mailto:Scott.Townsend@Phoenix.com) **Bob Hale** Intel Corporation 5200 N.E. Elam Young Parkway Hillsboro, OR 97124-6497 Phone: (503) 696-4249 Fax: (503) 648-6705 Email: [robert_p_hale@ccm2.hf.intel.com](mailto:robert_p_hale@ccm2.hf.intel.com) ## 1.2 Related Documents | Title | Author | Version | |------------------------------------|------------------------------|---------| | BIOS Boot Specification | Phoenix, Intel, Compaq | 1.01 | | Plug and Play BIOS Specification | Compaq, Phoenix, Intel | 1.0A | | EISA Specification | BCPR Services, Inc. | 3.12 | #################### File: BIOS%20Disassembly%20Ninjutsu%20Uncovered%201st%20Edition%20-%20Darmawan%20Salihun%20%28PDF%29%20POST_Memory_Manager_Specification.pdf Page: 4 Context: # 1.3 Terms ## Big Real Mode **Big Real Mode** is a modified version of the processor’s real mode with the segment limits changed from 1MB to 4GB. Big real mode allows the BIOS or an Option ROM to read and write extended memory without the overhead of protected mode. The BIOS puts the processor in big real mode during POST to allow simplified access to extended memory. The processor will be in big real mode while the PMM services are callable. ## BIOS The **Basic Input Output System** is the system software embedded on a chip located on the computer’s main board. The BIOS executes POST to test and initialize the system components and then boots the operating system. The BIOS also handles the low-level input/output to the various peripheral devices connected to the computer at runtime. Additionally, most BIOSes have a Setup program that allows the user to configure the system. ## Extended Memory The **Extended Memory** area starts at memory address 1MB and ends at memory address 4GB. Extended memory is normally only accessible when the processor is in protected mode. One exception to this is big real mode (see above). Section 3.2.4 provides guidelines as to how a PMM client may access extended memory. ## Gate A20 **Gate A20** controls 1MB memory wrap-around. When Gate A20 is enabled, it forces memory accesses to wrap around and fall within the 0-1MB area by forcing address line 20 to be zero. This has the effect of not allowing access to extended memory. When Gate A20 is disabled, memory accesses beyond 1MB do not wrap around, thus allowing access to all of extended memory. ## Option ROM An **Option ROM** (Read Only Memory) is a software component located in a ROM chip on an add-in card or the system board. Its physical address is in system memory between addresses C0000H and DFFFFH. The BIOS may copy the component to shadow memory during POST. An Option ROM is characterized by the first two locations containing a two-byte signature of 55h AAh. Option ROMs are responsible for initializing their associated hardware, allowing it to be available to the rest of the system for booting or runtime. ## Paragraph A **Paragraph** is 16 contiguous bytes of memory. Paragraph alignment of data means that the address of the data is of the form xxxx0h. ## PMM The **POST Memory Manager** is a software component of the BIOS that provides for the allocation of memory blocks during system POST. ## POST The **Power-On Self-Test** is the part of the BIOS that takes control immediately after the computer is turned on. POST initializes the computer hardware in preparation for loading the operating system. ## Run-Time Execution of run-time software takes place after the operating system has loaded. BIOS run-time services are available at POST and remain callable after the operating system has booted. Application programs and operating systems call BIOS run-time services for hardware-related functions. The PMM services are not callable during run-time, and buffers allocated by the PMM during POST are not available at runtime. #################### File: BIOS%20Disassembly%20Ninjutsu%20Uncovered%201st%20Edition%20-%20Darmawan%20Salihun%20%28PDF%29%20POST_Memory_Manager_Specification.pdf Page: 6 Context: # 3 PMM Services Interface Clients that access the PMM do so through a set of function calls collectively referred to as PMM Services. PMM Services reside within the system BIOS and are available to both internal BIOS clients as well as Option ROM clients once the presence of PMM Services has been detected. In a system BIOS implementation of the POST Memory Manager Specification, all three PMM Services function calls are required to be present. ## 3.1 Detecting PMM Services A data structure exists within the BIOS for PMM presence detection. The PMM Structure is located in the system BIOS address space on a paragraph boundary between segment addresses E000h and FFFFh. There will be only one PMM Structure in the system BIOS. The presence of the PMM Structure indicates that the PMM Services are present and available for calling. ### 3.1.1 PMM Structure | Offset | Name | Size | Value | Description | |--------|-------------------------|------|--------|------------------------------------| | 00h | SignatureByteOne | BYTE | ‘S’ | Signature byte 1. | | 01h | SignatureByteTwo | BYTE | ‘P’ | Signature byte 2. | | 02h | SignatureByteThree | BYTE | ‘M’ | Signature byte 3. | | 03h | SignatureByteFour | BYTE | ‘M’ | Signature byte 4. | | 04h | StructureRevision | BYTE | 01h | Structure revision. | | 05h | StructureLength | BYTE | Varies | Length of this structure in bytes. | | 06h | StructureChecksum | BYTE | Varies | Checksum update field. | | 07h | EntryPoint | FAR PTR | Varies | Segment:Offset of PMM Services entry point. | | 08h | Reserved | 5 BYTES | 0 | Reserved | The StructureRevision field will be incremented if the PMM Structure changes. All fields up to and including the EntryPoint field are guaranteed not to change location within the PMM Structure. Any change to the PMM Structure will be at or beyond the Reserved field only. ### 3.1.2 Detection Algorithm A client follows this procedure to locate and access PMM Services: 1. Search for the four-byte “SPMM” string on paragraph boundaries starting at E000h, and ending, if not found, at FFFFh. 2. Verify that the PMM Structure data is valid by performing a checksum. The checksum is calculated by doing a byte-wise sum of the entire PMM Structure and comparing this sum with zero. If the checksum is not zero, then the PMM Structure data is not valid and the EntryPoint field should not be called. 3. Optionally inspect the StructureRevision field to determine the appropriate structure map. The StructureRevision field changes if previously reserved fields in the PMM Structure are redefined to be valid fields. 4. Make calls to the EntryPoint field in the PMM Structure to allocate and free memory as desired. ## 3.2 PMM Services The functions described below define PMM Services. The calls are defined as if called from the C programming language. The PMM interface uses the standard C large model calling convention. In particular, return values of type unsigned long are returned in the DX:AX register pair. #################### File: BIOS%20Disassembly%20Ninjutsu%20Uncovered%201st%20Edition%20-%20Darmawan%20Salihun%20%28PDF%29%20POST_Memory_Manager_Specification.pdf Page: 7 Context: # 3.2.1 Interface Style The `EntryPoint` field in the PMM Structure points to the PMM Services interface. Clients call the PMM Services by executing a far call to the address specified in the `EntryPoint` field. PMM Services return to the client via a far return. Values returned to the caller are placed in the DX:AX register pair. The flags and all registers, other than DX and AX, are preserved across calls to PMM services. ## 3.2.2 Stack Requirements PMM Services requires a minimum stack size of 256 bytes. BIOSes that support the PMM must provide a stack size of at least 1KB when calling an initialization vector, a Boot Connection Vector, or a Bootstrap Entry Vector of a PCI or Plug ISA Option ROM. ## 3.2.3 Memory Block Alignment The default alignment of memory blocks is on a paragraph boundary. Alignment of a memory block on a particular memory boundary (larger than a paragraph) is supported by specifying the desired alignment in the `length` parameter of the `pmnAllocate` function. ## 3.2.4 Accessing Extended Memory This section specifies how clients should access extended memory blocks allocated by the PMM. When control is passed to an option ROM from a BIOS that supports PMM, the processor will be in real mode, and Gate A20 will be disabled (segment wrap turned off). This allows access to extended memory blocks using real mode addressing. In big real mode, access to memory above 1MB can be accomplished by using a 32-bit extended index register (EDI, etc.) and setting the segment register to 0000h. The following code example assumes that the `pmnAllocate` function was just called to allocate a block of extended memory, and DX:AX returned the 32-bit buffer address. ```assembly ; Assume here that DX:AX contains the 32-bit address of our allocated buffer. ; Clear the DS segment register. push 0000h pop ds ; Put the DX:AX 32-bit buffer address into EDI. mov di, dx ; Get the upper word. shl di, 16 ; Shift it to upper EDI. mov di, ax ; Get the lower word. ; Example: clear the first four bytes of the extended memory buffer. mov [edi], 00000000h ; D:EDI is used as the memory pointer. ``` In a similar way, the other segment registers and 32-bit index registers can be used for extended memory accessing. **WARNING:** Clients must not modify the state of Gate A20, put the processor in protected mode, or call BIOS Interrupt 15, functions 87h or 89h. Any of these actions could alter the big real mode of the processor and could lead to a catastrophic system failure. # 3.3 pmnAllocate - Function 0 ## 3.3.1 Description The `pmnAllocate` function attempts to allocate a memory block of the specified type and size, and returns the address of the memory block to the caller. The memory block is a contiguous array of paragraphs whose size is specified by the length parameter. The contents of the allocated memory block are undefined and must be initialized by the client. #################### File: BIOS%20Disassembly%20Ninjutsu%20Uncovered%201st%20Edition%20-%20Darmawan%20Salihun%20%28PDF%29%20POST_Memory_Manager_Specification.pdf Page: 8 Context: # 3.3.2 Function Prototype ```c unsigned long {entryPoint}( unsigned int function, // 0 for pmAllocate unsigned long length, // in paragraphs unsigned long handle, // handle to assign to memory block unsigned int flags // bit flags specifying options ); ``` ## 3.3.3 Parameters ### function Value for pmAllocate. Invalid values for the function parameter (0x0003…0xFFFF) cause an error value of 0xFFFFFFFF to be returned, signaling that the function is not supported. ### length The size of the requested memory block in paragraphs, or if length is 0x00000000, no memory is allocated and the value returned is the size of the largest memory block available for the memory type specified in the flags parameter. The alignment bit in the flags register is ignored when calculating the largest memory block available. ### handle A client-specified identifier to be associated with the allocated memory block. A handle of 0xFFFFFFFF indicates that no identifier should be associated with the block. Such a memory block is known as an “anonymous” memory block and cannot be found using the pmnFind function (see below). If a specified handle for a requested memory block is already used in a currently allocated memory block, the error value of 0x00000000 is returned. ### flags A bitmap used by the client to designate options regarding memory allocation. | Bits | Field | Value | Description | |-------|--------------|-------|-------------------------------------------------------| | 1.0 | MemoryType | 1.3 | 0 = Invalid | | | | | 1 = Requesting conventional memory block (0 to 1MB). | | | | | 2 = Requesting extended memory block (1MB to 4GB). | | | | | 3 = Requesting either a conventional or an extended memory block, whichever is available. | | 2 | Alignment | 0.1 | 0 = No alignment. | | | | | 1 = Use alignment from the length parameter. | | 15.3 | Reserved | 0 | Reserved for future expansion, must all be zero. | ### flags.MemoryType Specifies whether the requested memory block should be allocated from conventional memory, extended memory, or from either conventional or extended memory. At least one of the bits in this field must be set. If bit 0 is set, the PMM will attempt to allocate only from conventional memory. If bit 1 is set, the PMM will attempt to allocate only from extended memory. If both bits 0 and 1 are set, the PMM will allocate from either type. #################### File: BIOS%20Disassembly%20Ninjutsu%20Uncovered%201st%20Edition%20-%20Darmawan%20Salihun%20%28PDF%29%20POST_Memory_Manager_Specification.pdf Page: 9 Context: # Memory Management Functions ## 3.4 pmmFind - Function 1 ### 3.4.1 Description The `pmmFind` function returns the address of the memory block associated with the specified handle. The `entryPoint` identifier is of type function pointer, and is a far address. It must be assigned the value of the `EntryPoint` field in the PMM structure before it can be called. The return value is of type unsigned long and represents the 32-bit physical memory address of the memory block that is associated with the handle. A return value of `0x00000000` indicates that the handle does not correspond to a currently allocated memory block. ### 3.4.2 Function Prototype ```c unsigned long (*entryPoint) { unsigned int function, // 1 for pmmFind unsigned long handle // handle assigned to memory block }; ``` ### 3.4.3 Parameters - **function** 1 for `pmmFind`. Invalid values for the function parameter (`0x0003`...`0xFFFF`) cause an error value of `0xFFFFFFFF` to be returned, signaling that the function is not supported. - **handle** An identifier specified by the client that was assigned to a memory block when the `pmmAllocate` function was called. If called with an anonymous handle (`0xFFFFFFFF`) or a currently undefined handle, a value of `0x00000000` will be returned indicating that no associated memory block was found. ## 3.5 pmmDeallocate - Function 2 ### 3.5.1 Description The `pmmDeallocate` function frees the specified memory block that was previously allocated by `pmmAllocate`. Memory blocks are not cleared to all zeros by the PMM before being deallocated. Memory below 1MB is cleared by the BIOS before OS boot, but deallocated extended memory blocks in particular will not explicitly be cleared. Clients of the PMM should not rely on the contents of deallocated memory blocks. #################### File: BIOS%20Disassembly%20Ninjutsu%20Uncovered%201st%20Edition%20-%20Darmawan%20Salihun%20%28PDF%29%20POST_Memory_Manager_Specification.pdf Page: 10 Context: The entryPoint identifier is of type far function pointer. It must be assigned the value of the EntryPoint field in the PMM structure before it can be called. The return value is of type unsigned long. If the memory block was deallocated correctly, the return value is `0x00000000`. If there was an error, the return value is non-zero. The error return values are implementation dependent, so long as they are non-zero. ## 3.5.2 Function Prototype ```c unsigned long (*entryPoint) ( unsigned int function, // 2 for pmmDeallocate unsigned long buffer // value returned by pmnAllocate ); ``` ## 3.5.3 Parameters ### function 2 for `pmmDeallocate`. Invalid values for the function parameter (`0x0003...0xFFFF`) cause an error value of `0xFFFFFFFF` to be returned, signaling that the function is not supported. ### buffer The 32-bit physical address of the memory block. This is the value that was returned by the `pmmAllocate` function. #################### File: BIOS%20Disassembly%20Ninjutsu%20Uncovered%201st%20Edition%20-%20Darmawan%20Salihun%20%28PDF%29%20POST_Memory_Manager_Specification.pdf Page: 11 Context: # 3.6 C Language Coding Example The following C language code example illustrates how to use PMM Services. It finds the PMM entry point, allocates a 16KB conventional memory block, verifies the block is found by using its handle, and deallocates the memory block. No error checking is performed. ```c // Include library files: #include // Structure templates and type definitions: typedef unsigned char UCHAR; typedef unsigned int UINT; typedef unsigned long ULONG; typedef struct { ULONG signature; // PMM Structure UCHAR revision; UCHAR length; UCHAR checksum; UCHAR reserved[5]; } PMM_STRUCTURE; // Equate definitions: #define PMM_ALLOCATE 0 // PMM function numbers. #define PMM_FIND 1 #define PMM_DEALLOCATE 2 #define BUFFER_SIZE 1024 // Number of paragraphs for 16KB buffer. #define HANDLE 0x12345678 // Handle to test PMM. #define PMM_SIGNATURE 0x0ABCD5204 // "PMM" string in DWORD format. #define NULL_FUNCTION_PTR ((ULONG) 0) // Function declarations: void main(void); ULONG findPMMEntry(void); // Start of main program. void main(void) { ULONG (*pmmEntry[])() = { findPMMEntry(), result }; UCHAR buffer[BUFFER_SIZE], *buffer_alias; // Get PMM Services entry point, exit if not found. if ((pmmEntry[0] == NULL_FUNCTION_PTR)) exit(1); // Allocate a 16KB buffer and assign our own handle to it. buffer = (UCHAR *) (pmmEntry[PMM_ALLOCATE, BUFFER_SIZE, HANDLE]); // Look up the buffer address based on our known handle. buffer_alias = (UCHAR *) (pmmEntry[PMM_FIND, HANDLE]); // Deallocate the buffer. result = *(pmmEntry[PMM_DEALLOCATE, buffer]); } ``` #################### File: BIOS%20Disassembly%20Ninjutsu%20Uncovered%201st%20Edition%20-%20Darmawan%20Salihun%20%28PDF%29%20POST_Memory_Manager_Specification.pdf Page: 13 Context: ```markdown # 3.7 Assembly Language Coding Example The following assembly language coding example performs the equivalent of the C language example above. Note the use of x86 instructions (pushd, etc.) and 32-bit instruction operands. ```assembly ; Find the PM Services entry point and checksum the structure. call findPMEntry ; DS:SI will point to PM structure. ; Allocate a 16KB buffer and assign our own handle to it. push 0031h ; Specify conventional memory. push 12345678h ; Our handle is 12345678h. push 00000400h ; Buffer size is 16K (40th paragraph). push 0000h ; Specify allocate - function 0. call DWORD PTR [esi + 77] ; Call the entry point in the structure. add sp, 12 ; Clean up stack after call - C style. cmp dx, 0000h ; Buffer address is in DX:AX (32-bit). jbe allocSuccess jmp failed ; Return value of 00000000h is an error. allocSuccess: ; Save our buffer address from DX:AX into EDI. mov di, dx ; Put the upper word into DI. shl di, 16 ; Shift it to upper EDI. mov di, ax ; Put the lower word into EDI. ; Look up the buffer address based on our known handle. push 12345678h ; Our handle is 12345678h. push 0031h ; Specify find - function 1. call DWORD PTR [esi + 77] ; Call the entry point in the structure. add sp, 12 ; Clean up stack after call - C style. cmp dx, 0000h ; Buffer address is in DX:AX (32-bit). jbe findSuccess jmp findFailed ; Return value is in DX:AX (32-bit). findSuccess: push edi ; Retrieve buffer address from EDI. push 002h ; Specify deallocate - function 2. call DWORD PTR [esi + 77] ; Call the entry point in the structure. add sp, 12 ; Clean up stack after call - C style. cmp dx, 0000h ; Return value is in DX:AX (32-bit). jbe passed jmp failed ; Return value of 00000000h is an error. passed: ; No error occurred. ``` ``` #################### File: BIOS%20Disassembly%20Ninjutsu%20Uncovered%201st%20Edition%20-%20Darmawan%20Salihun%20%28PDF%29%20POST_Memory_Manager_Specification.pdf Page: 15 Context: # Disclaimer In cases where add-in cards and Option ROM code is purchased from one company for distribution by a different company, the exact definition of who is the manufacturer of a particular peripheral may become murky. It is up to the various parties involved to establish their own practices within the guidelines of this document. A case where this may arise is a vendor of Option ROMs selling those to several card manufacturers. Note that Option ROMs may allocate anonymous handles without fear of colliding with previously allocated handles. ## 4.2 Recommended Method for the Use of Named Blocks In general, only one named region is needed per card. The named block is allocated and used as a list of pointers to unnamed blocks, functioning somewhat like a directory in a file system. As an example, assume that the hard disk controller card marketed by XYZ has an Option ROM. During its initialization process, the card's Option ROM needs to allocate a buffer for reading from the hard disks it controls and a linked list of blocks describing each possible bootable hard disk. The following pseudo code would implement this scenario. (Error recovery, while important, is not shown for the sake of clarity.) ```c #define NoId 0xFFFFFFFF // The ID passed in for anon blocks. #define XYZ20000 ( ((‘Y’^A+1) << 26) | \ ((‘Y’^A+1) << 21) | \ ((‘Z’^A+1) << 16) ) #define ConvMem 1 // allocate blocks between 0 and 640k struct dir { byte *buffer; struct BootStruct *boot; }; struct dir *base, *t; // readability function unsigned long pmmlAllocate (unsigned long handle, size_t) { entryPoint = findPMEntry(); return (*entryPoint)(PMM_ALLOCATE, size, handle, ConvMem); } // (n+15)/16 calculates the required number of paragraphs given // an equal to the structure's length in bytes. base = pmmlAllocate(XYZ20000, (sizeof(dir)+15)/16); // allocate the buffer. base->buffEer = pmmlAllocate(NoId, (BufLen+15)/16); base->boot = NULL; for each device i { // do stuff if (device is bootable) { t = pmmlAllocate(NoId, (sizeof(BootStruct)+15)/16); t->next = base->boot; base->boot = t; } } ``` This means that, in general, only one named block is required per Option ROM. (The value of XYZ20000 is 0x633A0000.) Visually, the resulting data structure is: #################### File: BIOS%20Disassembly%20Ninjutsu%20Uncovered%201st%20Edition%20-%20Darmawan%20Salihun%20%28PDF%29%20POST_Memory_Manager_Specification.pdf Page: 16 Context: # Support of Multiple Structures With One Named Block ### XYZ0000 ``` Dir | Buffer | |--- BootStruct | |--- BootStruct ``` ### Key - **Anonymous Block** - **Named Block** - **Null Pointer** ## 4.3 Finding Other Cards in the System The use of PnP IDs for named handles has another consequence: it makes it possible to perform a simplified search for other similar cards in the system. Assume that XYZ's Option ROMs always attempt to allocate a block of memory named “XYZ0000”. If two of XYZ's Option ROMs exist in the same system, the second request to allocate will fail. If the company's Option ROMs instead do a `pmmFind` for the handle XYZ0000 prior to attempting to allocate memory with the same handle, the Option ROM can use the previously initialized allocation. In companies with more complex product lines, each set of products may need to use a different “family” ID (e.g., XYZ0000 for one family, XYZ0100 for another, etc.). Pseudocode implementing multiple card support is shown below. ```c #define NoId 0xFFFFFFFF #define XYZ0000 ((*(Y*++)+1) << 26) | \ ((*(Y*++)+1) << 21) | \ ((*(Y*++)+1) << 16) struct dir { struct dir *nextcard; // point to next directory byte *buffer; struct BootStruct *boot; int cardsCS; } dir; struct dir *base, *t; // pmmAllocate is assumed to be defined as above. if ((base = (*entryPoint)(PMM_FIND, XYZ0000)) != 0) { t = pmmAllocate(NoId, (sizeof(dir) + 15) / 16); t->nextcard = base->nextcard; base = t; } ``` #################### File: BIOS%20Disassembly%20Ninjutsu%20Uncovered%201st%20Edition%20-%20Darmawan%20Salihun%20%28PDF%29%20POST_Memory_Manager_Specification.pdf Page: 17 Context: ```markdown } else { base = pmAllocate(XYZ0000, (sizeof(dir)+15)/16); base->nextcard = NULL; } ... base->buffer = pmAllocate(NoId, (BufLen+15)/16); base->cardsCS = GetOurCodeSegment(); // base->boot = NULL; for each device i { // do stuff if (device is bootable) { t = pmAllocate(NoId, (sizeof(BootStruct)+15)/16); t->next = base->boot; base->boot = t; } } Visually, the resulting data structure looks like this: # Support of Multiple Cards With One Named Block ``` XYZ0000 ├── Dir │ ├── 0D000h (Card 1) └── Dir ├── 0D800h (Card 2) ├── Buffer └── Buffer ├── BootStruct └── BootStruct ``` ## Key - `[` Anonymous Block `]` - `---` Named Block - `->` Null Pointer [ End of Specification ] ``` #################### File: Project%20Oberon%20The%20Design%20of%20an%20Operating%20System%2C%20a%20Compiler%2C%20and%20a%20Computer%20-%20Niklaus%20Wirth%2C%20J%C3%BCrg%20Gutknecht%20%28PDF%29%20%20UsingOberon.pdf Page: 2 Context: # System Commands | Command | Description | |----------------|--------------------------------------| | System.Close | close the viewer | | Edit.Store | store the text as a file | | Edit.Search | search for the selected word from the position of the caret | ## Editing Texts The most frequent task is editing a text. For this case, some commands can be issued by simple mouse clicks alone rather than clicking the command name. When a text is to be entered, it is always inserted at the position of the caret displayed as a hook. The caret is positioned by pointing (with the cursor) at the desired position and clicking the left button (pointer button). Then the text is entered by simply typing it. A piece of text can be selected in order to be subjected to a subsequent command. This is done by moving the cursor over the piece of text while holding the right button (select button) pressed. Often it is only necessary to select the first character of a parameter. Selection can be turned into deletion by briefly clicking also the left button while holding the right button pressed. This is called inter-clicking. Similarly, a piece of text being selected can be copied over to the caret position by inter-clicking the middle button. The functions of the mouse buttons are summarized by the following table: | Inter-click | left | middle | right | |----------------|---------------------|---------------|------------------| | Primary click: | set caret | - | copy | | | command | - | - | | | select | delete | copy | The keyboard features two special keys, so-called control keys. The `Esc` key undoes selections. The `Backspace` key deletes the character left of the caret. Text viewers have a scroll bar at their left edge. While the cursor is in the scroll bar, it assumes the form of an up/down pointer to show its difference in functionality. Clicking a button then causes the text to scroll up or down. - **Left button:** the current line moves up to the top of the viewer - **Middle button:** shows the section at the relative position given by the cursor - **Right button:** the top line moves down to the current cursor position ## Viewer Handling Viewers can be extended, moved, or copied. A viewer is enlarged or shrunk by clicking the left button, while the cursor is in the title bar, and then dragging the bar up or down. A viewer is moved to another location by also inter-clicking with the middle button. A duplicate of a viewer may be generated by activating the command `System.Copy` in the title bar. Note that in this case the old and the new viewer show the same text, and not a copy of the text. This facility comes in handy when a piece of text needs to be #################### File: Project%20Oberon%20The%20Design%20of%20an%20Operating%20System%2C%20a%20Compiler%2C%20and%20a%20Computer%20-%20Niklaus%20Wirth%2C%20J%C3%BCrg%20Gutknecht%20%28PDF%29%20%20UsingOberon.pdf Page: 3 Context: # System Tool Commands The command `System.grow` in the title bar generates a copy extending over the entire column (or over the entire display). By closing the viewer (command `System.close` in the title bar), the original viewer reappears. We may imagine that `grow` lifts a viewer to an overlay in the third dimension. ## Commands The following commands appear in the standard tool `System.Tool`. The character `^` indicates that a name must be selected previously. The command then applies to that named text (file, as in the example above). The character `~` indicates that a list of names must be inserted before the `~` character, and the command will apply to all named objects. | Command | Description | |-------------------------|---------------------------------------------------------| | `System.Open ^` | open viewer in the system track to the right | | `System.Recall` | close the last viewer opened | | `Edit.Open ^` | open viewer in the user track to the left | | `Edit.Recall` | undo last editing operation | | `Edit.ChangeFont` | applies to selected piece of text | | `Edit.SetFont` | use specified font for subsequent input | | `System.Directory ^` | search directory for specified file names | | `System.Free ~` | remove specified modules from store | | `System.CopyFiles ->` | e.g. `file1 => file2 file3 => file4` | | `System.RenameFiles ->` | e.g. `file1 => file2 file3 => file4` | | `System.DeleteFiles ->` | e.g. `file1 file2 file3 ->` (from file directory) | | `System.ShowModules` | | | `System.ShowCommands ^`| compile selected text | | `Hilbert.Draw` | draw Hilbert curve, as an example | When clicking a command `M.P.`, module `M` is searched in the file store and, if found and not already present, loaded into main store. Then its command `P` is searched and executed (if found). A list of loaded modules can be generated by the command `System.ShowModules`, and a list of its commands is obtained by the command `System.ShowCommands`. Any parameter-less procedure in any (compiled) module is accessible as a command. Its parameters are accessed via a scanner. As an example, consider the following short program: ```pascal MODULE M0: IMPORT Texts, Oberon; VAR W: Texts.Writer; PROCEDURE P0; VAR sum: INTEGER; S: Texts.Scanner; BEGIN sum := 0; Texts.OpenScanner(S, Oberon.Par.text, Oberon.Par.pos); Texts.Scan(S); WHILE S.Class = Texts.Int DO Texts.Writeln(W, S.i, ':'); sum := sum + S.i; Texts.Scan(S); END; END; ``` #################### File: Project%20Oberon%20The%20Design%20of%20an%20Operating%20System%2C%20a%20Compiler%2C%20and%20a%20Computer%20-%20Niklaus%20Wirth%2C%20J%C3%BCrg%20Gutknecht%20%28PDF%29%20%20UsingOberon.pdf Page: 4 Context: ``` Texts.Write(W, sum, 8); Texts.WriteLn(W); Texts.Append(Oberon.Log, W.buf) END P0; BEGIN Texts.OpenWriter(W) END M0. After its successful compilation (ORP.Compile @), the command M0.PO 2 3 5 7 11 causes these numbers and their sum to be output to the standard viewer **System.Log**. ## The core of the Oberon System The system's core consists of a loop which continuously senses for a command to appear. The command is identified, control is dispatched, and the command is executed. A command may stem from an explicit click (middle button) on a text of the form M.P, or it may be a click of the left or right mouse buttons (see editing commands). A further source of input is the keyboard. If any key is pressed, this is interpreted as a command to read that character. Exceptions are the **esc**, **ctrl-z** (or **F1**), and **backspace** keys. **esc** is interpreted as a command to undo all selections, **backspace** to remove the character left of the caret and **ctrl-z** to set the global marker. ``` yes no mouse key? | keyboard? ``` The initially loaded system contains, apart from module Oberon, the command module **System**, a text system (modules TextFrames, MenuViewers, Texts, Fonts, Input), a viewer system (modules Viewers, Display), the loader and linker (module Modules), a file system (modules Files, FileDir), and the disk space manager and the garbage collector (module Kernel). The compiler is loaded on demand, like other application programs. [Project Oberon - ETH Zurich](https://www.inf.ethz.ch/personal/wirth/ProjectOberon/index.html) [Project Oberon](http://www.projectoberon.com/) ``` ########## """QUERY: Please summarize the whole context. It is important that you include a summary for each file. All files should be included, so please make sure to go through the entire context""" Consider the chat history for relevant information. Use all information included. Use as much tokens as needed. Important: If you find information separated by a | in the context, it is a table formatted in Markdown. Final Files Sources: Git%20Magic%20-%20Ben%20Lynn%2C%20et%20al.%20%28HTML%2C%20PDF%2C%20EPUB%29.pdf - Page 1, Git%20Magic%20-%20Ben%20Lynn%2C%20et%20al.%20%28HTML%2C%20PDF%2C%20EPUB%29.pdf - Page 2, Git%20Magic%20-%20Ben%20Lynn%2C%20et%20al.%20%28HTML%2C%20PDF%2C%20EPUB%29.pdf - Page 3, Git%20Magic%20-%20Ben%20Lynn%2C%20et%20al.%20%28HTML%2C%20PDF%2C%20EPUB%29.pdf - Page 4, Git%20Magic%20-%20Ben%20Lynn%2C%20et%20al.%20%28HTML%2C%20PDF%2C%20EPUB%29.pdf - Page 5, Git%20Magic%20-%20Ben%20Lynn%2C%20et%20al.%20%28HTML%2C%20PDF%2C%20EPUB%29.pdf - Page 6, Git%20Magic%20-%20Ben%20Lynn%2C%20et%20al.%20%28HTML%2C%20PDF%2C%20EPUB%29.pdf - Page 7, Git%20Magic%20-%20Ben%20Lynn%2C%20et%20al.%20%28HTML%2C%20PDF%2C%20EPUB%29.pdf - Page 8, Git%20Magic%20-%20Ben%20Lynn%2C%20et%20al.%20%28HTML%2C%20PDF%2C%20EPUB%29.pdf - Page 9, Git%20Magic%20-%20Ben%20Lynn%2C%20et%20al.%20%28HTML%2C%20PDF%2C%20EPUB%29.pdf - Page 10, Git%20Magic%20-%20Ben%20Lynn%2C%20et%20al.%20%28HTML%2C%20PDF%2C%20EPUB%29.pdf - Page 11, Git%20Magic%20-%20Ben%20Lynn%2C%20et%20al.%20%28HTML%2C%20PDF%2C%20EPUB%29.pdf - Page 13, Git%20Magic%20-%20Ben%20Lynn%2C%20et%20al.%20%28HTML%2C%20PDF%2C%20EPUB%29.pdf - Page 14, Git%20Magic%20-%20Ben%20Lynn%2C%20et%20al.%20%28HTML%2C%20PDF%2C%20EPUB%29.pdf - Page 15, Git%20Magic%20-%20Ben%20Lynn%2C%20et%20al.%20%28HTML%2C%20PDF%2C%20EPUB%29.pdf - Page 16, Git%20Magic%20-%20Ben%20Lynn%2C%20et%20al.%20%28HTML%2C%20PDF%2C%20EPUB%29.pdf - Page 17, Git%20Magic%20-%20Ben%20Lynn%2C%20et%20al.%20%28HTML%2C%20PDF%2C%20EPUB%29.pdf - Page 18, Git%20Magic%20-%20Ben%20Lynn%2C%20et%20al.%20%28HTML%2C%20PDF%2C%20EPUB%29.pdf - Page 19, Git%20Magic%20-%20Ben%20Lynn%2C%20et%20al.%20%28HTML%2C%20PDF%2C%20EPUB%29.pdf - Page 20, Git%20Magic%20-%20Ben%20Lynn%2C%20et%20al.%20%28HTML%2C%20PDF%2C%20EPUB%29.pdf - Page 21, Git%20Magic%20-%20Ben%20Lynn%2C%20et%20al.%20%28HTML%2C%20PDF%2C%20EPUB%29.pdf - Page 22, Git%20Magic%20-%20Ben%20Lynn%2C%20et%20al.%20%28HTML%2C%20PDF%2C%20EPUB%29.pdf - Page 23, Git%20Magic%20-%20Ben%20Lynn%2C%20et%20al.%20%28HTML%2C%20PDF%2C%20EPUB%29.pdf - Page 24, Git%20Magic%20-%20Ben%20Lynn%2C%20et%20al.%20%28HTML%2C%20PDF%2C%20EPUB%29.pdf - Page 25, Git%20Magic%20-%20Ben%20Lynn%2C%20et%20al.%20%28HTML%2C%20PDF%2C%20EPUB%29.pdf - Page 26, Git%20Magic%20-%20Ben%20Lynn%2C%20et%20al.%20%28HTML%2C%20PDF%2C%20EPUB%29.pdf - Page 27, Git%20Magic%20-%20Ben%20Lynn%2C%20et%20al.%20%28HTML%2C%20PDF%2C%20EPUB%29.pdf - Page 28, Git%20Magic%20-%20Ben%20Lynn%2C%20et%20al.%20%28HTML%2C%20PDF%2C%20EPUB%29.pdf - Page 29, Git%20Magic%20-%20Ben%20Lynn%2C%20et%20al.%20%28HTML%2C%20PDF%2C%20EPUB%29.pdf - Page 30, Git%20Magic%20-%20Ben%20Lynn%2C%20et%20al.%20%28HTML%2C%20PDF%2C%20EPUB%29.pdf - Page 31, Git%20Magic%20-%20Ben%20Lynn%2C%20et%20al.%20%28HTML%2C%20PDF%2C%20EPUB%29.pdf - Page 32, Git%20Magic%20-%20Ben%20Lynn%2C%20et%20al.%20%28HTML%2C%20PDF%2C%20EPUB%29.pdf - Page 33, Git%20Magic%20-%20Ben%20Lynn%2C%20et%20al.%20%28HTML%2C%20PDF%2C%20EPUB%29.pdf - Page 34, Git%20Magic%20-%20Ben%20Lynn%2C%20et%20al.%20%28HTML%2C%20PDF%2C%20EPUB%29.pdf - Page 35, Git%20Magic%20-%20Ben%20Lynn%2C%20et%20al.%20%28HTML%2C%20PDF%2C%20EPUB%29.pdf - Page 36, Git%20Magic%20-%20Ben%20Lynn%2C%20et%20al.%20%28HTML%2C%20PDF%2C%20EPUB%29.pdf - Page 37, Git%20Magic%20-%20Ben%20Lynn%2C%20et%20al.%20%28HTML%2C%20PDF%2C%20EPUB%29.pdf - Page 38, Git%20Magic%20-%20Ben%20Lynn%2C%20et%20al.%20%28HTML%2C%20PDF%2C%20EPUB%29.pdf - Page 41, Git%20Magic%20-%20Ben%20Lynn%2C%20et%20al.%20%28HTML%2C%20PDF%2C%20EPUB%29.pdf - Page 43, Git%20Magic%20-%20Ben%20Lynn%2C%20et%20al.%20%28HTML%2C%20PDF%2C%20EPUB%29.pdf - Page 44, Project%20Oberon%20The%20Design%20of%20an%20Operating%20System%2C%20a%20Compiler%2C%20and%20a%20Computer%20-%20Niklaus%20Wirth%2C%20J%C3%BCrg%20Gutknecht%20%28PDF%29%20PO.Applications.pdf - Page 1, Project%20Oberon%20The%20Design%20of%20an%20Operating%20System%2C%20a%20Compiler%2C%20and%20a%20Computer%20-%20Niklaus%20Wirth%2C%20J%C3%BCrg%20Gutknecht%20%28PDF%29%20PO.Applications.pdf - Page 3, Project%20Oberon%20The%20Design%20of%20an%20Operating%20System%2C%20a%20Compiler%2C%20and%20a%20Computer%20-%20Niklaus%20Wirth%2C%20J%C3%BCrg%20Gutknecht%20%28PDF%29%20PO.Applications.pdf - Page 4, Project%20Oberon%20The%20Design%20of%20an%20Operating%20System%2C%20a%20Compiler%2C%20and%20a%20Computer%20-%20Niklaus%20Wirth%2C%20J%C3%BCrg%20Gutknecht%20%28PDF%29%20PO.Applications.pdf - Page 5, Project%20Oberon%20The%20Design%20of%20an%20Operating%20System%2C%20a%20Compiler%2C%20and%20a%20Computer%20-%20Niklaus%20Wirth%2C%20J%C3%BCrg%20Gutknecht%20%28PDF%29%20PO.Applications.pdf - Page 6, Project%20Oberon%20The%20Design%20of%20an%20Operating%20System%2C%20a%20Compiler%2C%20and%20a%20Computer%20-%20Niklaus%20Wirth%2C%20J%C3%BCrg%20Gutknecht%20%28PDF%29%20PO.Applications.pdf - Page 8, Project%20Oberon%20The%20Design%20of%20an%20Operating%20System%2C%20a%20Compiler%2C%20and%20a%20Computer%20-%20Niklaus%20Wirth%2C%20J%C3%BCrg%20Gutknecht%20%28PDF%29%20PO.Applications.pdf - Page 9, Project%20Oberon%20The%20Design%20of%20an%20Operating%20System%2C%20a%20Compiler%2C%20and%20a%20Computer%20-%20Niklaus%20Wirth%2C%20J%C3%BCrg%20Gutknecht%20%28PDF%29%20PO.Applications.pdf - Page 11, Project%20Oberon%20The%20Design%20of%20an%20Operating%20System%2C%20a%20Compiler%2C%20and%20a%20Computer%20-%20Niklaus%20Wirth%2C%20J%C3%BCrg%20Gutknecht%20%28PDF%29%20PO.Applications.pdf - Page 12, Project%20Oberon%20The%20Design%20of%20an%20Operating%20System%2C%20a%20Compiler%2C%20and%20a%20Computer%20-%20Niklaus%20Wirth%2C%20J%C3%BCrg%20Gutknecht%20%28PDF%29%20PO.Applications.pdf - Page 13, Project%20Oberon%20The%20Design%20of%20an%20Operating%20System%2C%20a%20Compiler%2C%20and%20a%20Computer%20-%20Niklaus%20Wirth%2C%20J%C3%BCrg%20Gutknecht%20%28PDF%29%20PO.Applications.pdf - Page 14, Project%20Oberon%20The%20Design%20of%20an%20Operating%20System%2C%20a%20Compiler%2C%20and%20a%20Computer%20-%20Niklaus%20Wirth%2C%20J%C3%BCrg%20Gutknecht%20%28PDF%29%20PO.Applications.pdf - Page 15, Project%20Oberon%20The%20Design%20of%20an%20Operating%20System%2C%20a%20Compiler%2C%20and%20a%20Computer%20-%20Niklaus%20Wirth%2C%20J%C3%BCrg%20Gutknecht%20%28PDF%29%20PO.Applications.pdf - Page 16, Project%20Oberon%20The%20Design%20of%20an%20Operating%20System%2C%20a%20Compiler%2C%20and%20a%20Computer%20-%20Niklaus%20Wirth%2C%20J%C3%BCrg%20Gutknecht%20%28PDF%29%20PO.Applications.pdf - Page 17, Project%20Oberon%20The%20Design%20of%20an%20Operating%20System%2C%20a%20Compiler%2C%20and%20a%20Computer%20-%20Niklaus%20Wirth%2C%20J%C3%BCrg%20Gutknecht%20%28PDF%29%20PO.Applications.pdf - Page 18, Project%20Oberon%20The%20Design%20of%20an%20Operating%20System%2C%20a%20Compiler%2C%20and%20a%20Computer%20-%20Niklaus%20Wirth%2C%20J%C3%BCrg%20Gutknecht%20%28PDF%29%20PO.Applications.pdf - Page 19, Project%20Oberon%20The%20Design%20of%20an%20Operating%20System%2C%20a%20Compiler%2C%20and%20a%20Computer%20-%20Niklaus%20Wirth%2C%20J%C3%BCrg%20Gutknecht%20%28PDF%29%20PO.Applications.pdf - Page 20, Project%20Oberon%20The%20Design%20of%20an%20Operating%20System%2C%20a%20Compiler%2C%20and%20a%20Computer%20-%20Niklaus%20Wirth%2C%20J%C3%BCrg%20Gutknecht%20%28PDF%29%20PO.Applications.pdf - Page 21, Project%20Oberon%20The%20Design%20of%20an%20Operating%20System%2C%20a%20Compiler%2C%20and%20a%20Computer%20-%20Niklaus%20Wirth%2C%20J%C3%BCrg%20Gutknecht%20%28PDF%29%20PO.Applications.pdf - Page 22, Project%20Oberon%20The%20Design%20of%20an%20Operating%20System%2C%20a%20Compiler%2C%20and%20a%20Computer%20-%20Niklaus%20Wirth%2C%20J%C3%BCrg%20Gutknecht%20%28PDF%29%20PO.Applications.pdf - Page 23, Project%20Oberon%20The%20Design%20of%20an%20Operating%20System%2C%20a%20Compiler%2C%20and%20a%20Computer%20-%20Niklaus%20Wirth%2C%20J%C3%BCrg%20Gutknecht%20%28PDF%29%20PO.Applications.pdf - Page 24, Project%20Oberon%20The%20Design%20of%20an%20Operating%20System%2C%20a%20Compiler%2C%20and%20a%20Computer%20-%20Niklaus%20Wirth%2C%20J%C3%BCrg%20Gutknecht%20%28PDF%29%20PO.Applications.pdf - Page 25, Project%20Oberon%20The%20Design%20of%20an%20Operating%20System%2C%20a%20Compiler%2C%20and%20a%20Computer%20-%20Niklaus%20Wirth%2C%20J%C3%BCrg%20Gutknecht%20%28PDF%29%20PO.Applications.pdf - Page 26, Project%20Oberon%20The%20Design%20of%20an%20Operating%20System%2C%20a%20Compiler%2C%20and%20a%20Computer%20-%20Niklaus%20Wirth%2C%20J%C3%BCrg%20Gutknecht%20%28PDF%29%20PO.Applications.pdf - Page 27, Project%20Oberon%20The%20Design%20of%20an%20Operating%20System%2C%20a%20Compiler%2C%20and%20a%20Computer%20-%20Niklaus%20Wirth%2C%20J%C3%BCrg%20Gutknecht%20%28PDF%29%20PO.Applications.pdf - Page 28, Project%20Oberon%20The%20Design%20of%20an%20Operating%20System%2C%20a%20Compiler%2C%20and%20a%20Computer%20-%20Niklaus%20Wirth%2C%20J%C3%BCrg%20Gutknecht%20%28PDF%29%20PO.Applications.pdf - Page 29, Project%20Oberon%20The%20Design%20of%20an%20Operating%20System%2C%20a%20Compiler%2C%20and%20a%20Computer%20-%20Niklaus%20Wirth%2C%20J%C3%BCrg%20Gutknecht%20%28PDF%29%20PO.Applications.pdf - Page 30, Project%20Oberon%20The%20Design%20of%20an%20Operating%20System%2C%20a%20Compiler%2C%20and%20a%20Computer%20-%20Niklaus%20Wirth%2C%20J%C3%BCrg%20Gutknecht%20%28PDF%29%20PO.Applications.pdf - Page 31, Project%20Oberon%20The%20Design%20of%20an%20Operating%20System%2C%20a%20Compiler%2C%20and%20a%20Computer%20-%20Niklaus%20Wirth%2C%20J%C3%BCrg%20Gutknecht%20%28PDF%29%20PO.Applications.pdf - Page 32, Project%20Oberon%20The%20Design%20of%20an%20Operating%20System%2C%20a%20Compiler%2C%20and%20a%20Computer%20-%20Niklaus%20Wirth%2C%20J%C3%BCrg%20Gutknecht%20%28PDF%29%20PO.Applications.pdf - Page 33, Project%20Oberon%20The%20Design%20of%20an%20Operating%20System%2C%20a%20Compiler%2C%20and%20a%20Computer%20-%20Niklaus%20Wirth%2C%20J%C3%BCrg%20Gutknecht%20%28PDF%29%20PO.Applications.pdf - Page 34, Project%20Oberon%20The%20Design%20of%20an%20Operating%20System%2C%20a%20Compiler%2C%20and%20a%20Computer%20-%20Niklaus%20Wirth%2C%20J%C3%BCrg%20Gutknecht%20%28PDF%29%20PO.Applications.pdf - Page 35, Project%20Oberon%20The%20Design%20of%20an%20Operating%20System%2C%20a%20Compiler%2C%20and%20a%20Computer%20-%20Niklaus%20Wirth%2C%20J%C3%BCrg%20Gutknecht%20%28PDF%29%20PO.Applications.pdf - Page 36, Project%20Oberon%20The%20Design%20of%20an%20Operating%20System%2C%20a%20Compiler%2C%20and%20a%20Computer%20-%20Niklaus%20Wirth%2C%20J%C3%BCrg%20Gutknecht%20%28PDF%29%20PO.Applications.pdf - Page 37, Project%20Oberon%20The%20Design%20of%20an%20Operating%20System%2C%20a%20Compiler%2C%20and%20a%20Computer%20-%20Niklaus%20Wirth%2C%20J%C3%BCrg%20Gutknecht%20%28PDF%29%20PO.Applications.pdf - Page 38, Project%20Oberon%20The%20Design%20of%20an%20Operating%20System%2C%20a%20Compiler%2C%20and%20a%20Computer%20-%20Niklaus%20Wirth%2C%20J%C3%BCrg%20Gutknecht%20%28PDF%29%20PO.Applications.pdf - Page 39, Project%20Oberon%20The%20Design%20of%20an%20Operating%20System%2C%20a%20Compiler%2C%20and%20a%20Computer%20-%20Niklaus%20Wirth%2C%20J%C3%BCrg%20Gutknecht%20%28PDF%29%20PO.Applications.pdf - Page 40, Project%20Oberon%20The%20Design%20of%20an%20Operating%20System%2C%20a%20Compiler%2C%20and%20a%20Computer%20-%20Niklaus%20Wirth%2C%20J%C3%BCrg%20Gutknecht%20%28PDF%29%20PO.Applications.pdf - Page 41, Project%20Oberon%20The%20Design%20of%20an%20Operating%20System%2C%20a%20Compiler%2C%20and%20a%20Computer%20-%20Niklaus%20Wirth%2C%20J%C3%BCrg%20Gutknecht%20%28PDF%29%20PO.Applications.pdf - Page 42, Project%20Oberon%20The%20Design%20of%20an%20Operating%20System%2C%20a%20Compiler%2C%20and%20a%20Computer%20-%20Niklaus%20Wirth%2C%20J%C3%BCrg%20Gutknecht%20%28PDF%29%20PO.Applications.pdf - Page 43, Project%20Oberon%20The%20Design%20of%20an%20Operating%20System%2C%20a%20Compiler%2C%20and%20a%20Computer%20-%20Niklaus%20Wirth%2C%20J%C3%BCrg%20Gutknecht%20%28PDF%29%20PO.Applications.pdf - Page 44, Project%20Oberon%20The%20Design%20of%20an%20Operating%20System%2C%20a%20Compiler%2C%20and%20a%20Computer%20-%20Niklaus%20Wirth%2C%20J%C3%BCrg%20Gutknecht%20%28PDF%29%20PO.Applications.pdf - Page 45, Project%20Oberon%20The%20Design%20of%20an%20Operating%20System%2C%20a%20Compiler%2C%20and%20a%20Computer%20-%20Niklaus%20Wirth%2C%20J%C3%BCrg%20Gutknecht%20%28PDF%29%20PO.Applications.pdf - Page 46, Project%20Oberon%20The%20Design%20of%20an%20Operating%20System%2C%20a%20Compiler%2C%20and%20a%20Computer%20-%20Niklaus%20Wirth%2C%20J%C3%BCrg%20Gutknecht%20%28PDF%29%20PO.Applications.pdf - Page 47, Project%20Oberon%20The%20Design%20of%20an%20Operating%20System%2C%20a%20Compiler%2C%20and%20a%20Computer%20-%20Niklaus%20Wirth%2C%20J%C3%BCrg%20Gutknecht%20%28PDF%29%20PO.Applications.pdf - Page 48, Project%20Oberon%20The%20Design%20of%20an%20Operating%20System%2C%20a%20Compiler%2C%20and%20a%20Computer%20-%20Niklaus%20Wirth%2C%20J%C3%BCrg%20Gutknecht%20%28PDF%29%20PO.Applications.pdf - Page 49, Project%20Oberon%20The%20Design%20of%20an%20Operating%20System%2C%20a%20Compiler%2C%20and%20a%20Computer%20-%20Niklaus%20Wirth%2C%20J%C3%BCrg%20Gutknecht%20%28PDF%29%20PO.Applications.pdf - Page 50, Project%20Oberon%20The%20Design%20of%20an%20Operating%20System%2C%20a%20Compiler%2C%20and%20a%20Computer%20-%20Niklaus%20Wirth%2C%20J%C3%BCrg%20Gutknecht%20%28PDF%29%20PO.Applications.pdf - Page 51, Project%20Oberon%20The%20Design%20of%20an%20Operating%20System%2C%20a%20Compiler%2C%20and%20a%20Computer%20-%20Niklaus%20Wirth%2C%20J%C3%BCrg%20Gutknecht%20%28PDF%29%20PO.Applications.pdf - Page 52, Project%20Oberon%20The%20Design%20of%20an%20Operating%20System%2C%20a%20Compiler%2C%20and%20a%20Computer%20-%20Niklaus%20Wirth%2C%20J%C3%BCrg%20Gutknecht%20%28PDF%29%20PO.Applications.pdf - Page 53, Project%20Oberon%20The%20Design%20of%20an%20Operating%20System%2C%20a%20Compiler%2C%20and%20a%20Computer%20-%20Niklaus%20Wirth%2C%20J%C3%BCrg%20Gutknecht%20%28PDF%29%20PO.Applications.pdf - Page 54, Project%20Oberon%20The%20Design%20of%20an%20Operating%20System%2C%20a%20Compiler%2C%20and%20a%20Computer%20-%20Niklaus%20Wirth%2C%20J%C3%BCrg%20Gutknecht%20%28PDF%29%20PO.Applications.pdf - Page 55, Project%20Oberon%20The%20Design%20of%20an%20Operating%20System%2C%20a%20Compiler%2C%20and%20a%20Computer%20-%20Niklaus%20Wirth%2C%20J%C3%BCrg%20Gutknecht%20%28PDF%29%20PO.Applications.pdf - Page 57, Project%20Oberon%20The%20Design%20of%20an%20Operating%20System%2C%20a%20Compiler%2C%20and%20a%20Computer%20-%20Niklaus%20Wirth%2C%20J%C3%BCrg%20Gutknecht%20%28PDF%29%20PO.Applications.pdf - Page 58, Project%20Oberon%20The%20Design%20of%20an%20Operating%20System%2C%20a%20Compiler%2C%20and%20a%20Computer%20-%20Niklaus%20Wirth%2C%20J%C3%BCrg%20Gutknecht%20%28PDF%29%20PO.Applications.pdf - Page 62, Project%20Oberon%20The%20Design%20of%20an%20Operating%20System%2C%20a%20Compiler%2C%20and%20a%20Computer%20-%20Niklaus%20Wirth%2C%20J%C3%BCrg%20Gutknecht%20%28PDF%29%20PO.Applications.pdf - Page 63, Project%20Oberon%20The%20Design%20of%20an%20Operating%20System%2C%20a%20Compiler%2C%20and%20a%20Computer%20-%20Niklaus%20Wirth%2C%20J%C3%BCrg%20Gutknecht%20%28PDF%29%20PO.Applications.pdf - Page 65, Project%20Oberon%20The%20Design%20of%20an%20Operating%20System%2C%20a%20Compiler%2C%20and%20a%20Computer%20-%20Niklaus%20Wirth%2C%20J%C3%BCrg%20Gutknecht%20%28PDF%29%20PO.Applications.pdf - Page 67, Project%20Oberon%20The%20Design%20of%20an%20Operating%20System%2C%20a%20Compiler%2C%20and%20a%20Computer%20-%20Niklaus%20Wirth%2C%20J%C3%BCrg%20Gutknecht%20%28PDF%29%20PO.Applications.pdf - Page 68, Project%20Oberon%20The%20Design%20of%20an%20Operating%20System%2C%20a%20Compiler%2C%20and%20a%20Computer%20-%20Niklaus%20Wirth%2C%20J%C3%BCrg%20Gutknecht%20%28PDF%29%20PO.Applications.pdf - Page 80, Project%20Oberon%20The%20Design%20of%20an%20Operating%20System%2C%20a%20Compiler%2C%20and%20a%20Computer%20-%20Niklaus%20Wirth%2C%20J%C3%BCrg%20Gutknecht%20%28PDF%29%20PO.Applications.pdf - Page 81, Project%20Oberon%20The%20Design%20of%20an%20Operating%20System%2C%20a%20Compiler%2C%20and%20a%20Computer%20-%20Niklaus%20Wirth%2C%20J%C3%BCrg%20Gutknecht%20%28PDF%29%20PO.Applications.pdf - Page 84, Project%20Oberon%20The%20Design%20of%20an%20Operating%20System%2C%20a%20Compiler%2C%20and%20a%20Computer%20-%20Niklaus%20Wirth%2C%20J%C3%BCrg%20Gutknecht%20%28PDF%29%20PO.Applications.pdf - Page 85, BIOS%20Disassembly%20Ninjutsu%20Uncovered%201st%20Edition%20-%20Darmawan%20Salihun%20%28PDF%29%20POST_Memory_Manager_Specification.pdf - Page 1, BIOS%20Disassembly%20Ninjutsu%20Uncovered%201st%20Edition%20-%20Darmawan%20Salihun%20%28PDF%29%20POST_Memory_Manager_Specification.pdf - Page 2, BIOS%20Disassembly%20Ninjutsu%20Uncovered%201st%20Edition%20-%20Darmawan%20Salihun%20%28PDF%29%20POST_Memory_Manager_Specification.pdf - Page 3, BIOS%20Disassembly%20Ninjutsu%20Uncovered%201st%20Edition%20-%20Darmawan%20Salihun%20%28PDF%29%20POST_Memory_Manager_Specification.pdf - Page 4, BIOS%20Disassembly%20Ninjutsu%20Uncovered%201st%20Edition%20-%20Darmawan%20Salihun%20%28PDF%29%20POST_Memory_Manager_Specification.pdf - Page 6, BIOS%20Disassembly%20Ninjutsu%20Uncovered%201st%20Edition%20-%20Darmawan%20Salihun%20%28PDF%29%20POST_Memory_Manager_Specification.pdf - Page 7, BIOS%20Disassembly%20Ninjutsu%20Uncovered%201st%20Edition%20-%20Darmawan%20Salihun%20%28PDF%29%20POST_Memory_Manager_Specification.pdf - Page 8, BIOS%20Disassembly%20Ninjutsu%20Uncovered%201st%20Edition%20-%20Darmawan%20Salihun%20%28PDF%29%20POST_Memory_Manager_Specification.pdf - Page 9, BIOS%20Disassembly%20Ninjutsu%20Uncovered%201st%20Edition%20-%20Darmawan%20Salihun%20%28PDF%29%20POST_Memory_Manager_Specification.pdf - Page 10, BIOS%20Disassembly%20Ninjutsu%20Uncovered%201st%20Edition%20-%20Darmawan%20Salihun%20%28PDF%29%20POST_Memory_Manager_Specification.pdf - Page 11, BIOS%20Disassembly%20Ninjutsu%20Uncovered%201st%20Edition%20-%20Darmawan%20Salihun%20%28PDF%29%20POST_Memory_Manager_Specification.pdf - Page 13, BIOS%20Disassembly%20Ninjutsu%20Uncovered%201st%20Edition%20-%20Darmawan%20Salihun%20%28PDF%29%20POST_Memory_Manager_Specification.pdf - Page 15, BIOS%20Disassembly%20Ninjutsu%20Uncovered%201st%20Edition%20-%20Darmawan%20Salihun%20%28PDF%29%20POST_Memory_Manager_Specification.pdf - Page 16, BIOS%20Disassembly%20Ninjutsu%20Uncovered%201st%20Edition%20-%20Darmawan%20Salihun%20%28PDF%29%20POST_Memory_Manager_Specification.pdf - Page 17, Project%20Oberon%20The%20Design%20of%20an%20Operating%20System%2C%20a%20Compiler%2C%20and%20a%20Computer%20-%20Niklaus%20Wirth%2C%20J%C3%BCrg%20Gutknecht%20%28PDF%29%20%20UsingOberon.pdf - Page 2, Project%20Oberon%20The%20Design%20of%20an%20Operating%20System%2C%20a%20Compiler%2C%20and%20a%20Computer%20-%20Niklaus%20Wirth%2C%20J%C3%BCrg%20Gutknecht%20%28PDF%29%20%20UsingOberon.pdf - Page 3, Project%20Oberon%20The%20Design%20of%20an%20Operating%20System%2C%20a%20Compiler%2C%20and%20a%20Computer%20-%20Niklaus%20Wirth%2C%20J%C3%BCrg%20Gutknecht%20%28PDF%29%20%20UsingOberon.pdf - Page 4 ================================================== **Elapsed Time: 35.54 seconds** ================================================== FINAL ANSWER Answer: ================================================== **Elapsed Time: 0.00 seconds** ==================================================