\documentclass[a4paper,12pt]{article} \usepackage{geometry} \usepackage{titling} \usepackage{titlesec} \usepackage[english]{babel} \usepackage[hidelinks]{hyperref} \usepackage{listings} \usepackage{xcolor} \usepackage{graphicx} \usepackage[export]{adjustbox} \usepackage{forest} \usepackage{tikz-qtree} \usepackage{bchart} \usepackage[T1]{fontenc} \titleformat{\section}{\Huge}{}{0em}{}[\titlerule] \geometry{a4paper, total={170mm,257mm}, left=30mm, right=30mm} \author{Lucas Standen} \title{A parting gift of my knowledge} \begin{document} \maketitle \newpage \tableofcontents \setlength{\parskip}{1em} {\setlength{\parindent}{0cm} \section{Headnote} I leave this as a gift and thanks for all the things I learnt in your class, hopefully you can learn something from this. This document contains almost all the things I know about computer science; please enjoy! Not much of this will be stuff needed to teach the course but I still thought it would be nice to give you a wealth knowledge for the fun of it; If knowing me for 2 years wasn't enough tell, doing things for the fun of it is the best way to learn (in my opinion). I will be laying this document out into sections on general topics and then subsections of specifics, see the table of contents to do further reading. \section{The Unix wizard handbook} \subsection{The history} Unix was originally made at AT and T's bell labs, their research centre at the time (It belongs to Nokia now). It was designed as a small and lightweight OS for the 'weaker' computer at the site although it still took up and entire room. It was made by Ken Thompson and Denis Richie in the early 70's. They built everything from the ground up for the system, it was written in C, which they made specifically to write the system. It is often cited, but we will never truly know, that Ken was the person with all the ideas, and Denis was very good at implementing them, so most of the code in the original Unix was written by Denis. I don't know if there is anything relevant in that, but I thought it was interesting. Unix was sold to many university's in the late 70's and early 80's, and it was through this that it really started to gain some traction with the students. Many developed Unix further, because unlike modern paid software, Unix was distributed in source form, meaning you could tinker it as much as you liked, assuming you had brought a license. One of the main outcome of this would be BSD (The Bearkly software distribution) version of Unix. The people who worked on this system later went on to develop Java (All of computer science is so interlinked, its a small world!). In the 80/90's Unix was de facto in Uni's but the free software movement was starting to gain traction, and many re-writes or clones of Unix were starting to show up, like GNU (fun fact, GNU is a recursive acronym for GNU's not Unix!) \subsection{The philosophy} The main idea's behind Unix were that it needed to be small to run on the 'weaker' computer at bell labs. One of the leading limitations with this system was its small storage size of about 20Mb. To avoid this being an issue they came up with the following philosophy to ensure all software remained small, and simple even on the weaker computer. \begin{description} \item[Small reusable tools] The idea was that if you made one small program that did one thing and provided the ability to combine programs easily, you could let the user put together it together to make the tools they needed to their own needs. \item[Everything is a file] This was the idea that if everything is a file, from talking between systems, to talking to kernel, you could avoid many different APIs that would repetitive, instead all you needed was one single file and you could work with the entire system. \end{description} Something that was more unintentional but still worth noting is the Unix was designed for Teletypes, (paper printers that talked to computers), that didn't have the ability to overwrite sections of the page, and printed at a slow 10 characters per second. This effects a large amount of the user interfaces for Unix. \subsubsection{GNU/Linux} One of the many outcomes of the free software movement, it quickly became the easiest way for anyone to use Unix, releasing in 1991. There were open and free versions available at the time, such as BSD, however you still needed to be in a university to get a copy, as the internet wasn't really active at this time. It is ironic that Linux became a core part of this system, as it was in fact a small hobby project that the creator Linus Torvalds said 'this is just a small thing, it won't go anywhere'. The GNU part of GNU/Linux is the user land utility's of the system. Linux is only the kernel of the system, handling the low level system, such as the file system, memory allocation and process scheduler, the rest of the GNU/Linux system is the GNU core utils, such as the GNU bash shell. \subsection{The shell} \subsubsection{The interactive shell} The shell is the part of the system that you time commands into, it's a whole programming language that can run line by line (if it helps you think about it, its like using the python shell in IDLE). The shell was designed to be a helpful interface to take the user in and out of programs, it is fast and simple. So here is a whistle stop tour of all the things it does. The shell can store variables that contain strings, these strings can then be accessed by any program that is run in the same shell that the variable is in (think of it like a parent-child thing, the child process knows all the same things as the parent shell). The can be created like so 'myvar="hello"' and then can be accessed using a '\$' like so '\$myvar', this syntax is also seen in Perl and php, which are often compared to the Unix shell as they are all interpreted scripting languages. On a Unix system applications are stored in /bin, /usr/bin, /usr/local/bin, and \url{~/.local/bin} (and sometime other places too). The shell will allow you to type a commands name, like 'cp' (copy paste), and it will search through all the known locations of programs and run the first one it finds. The shell stores all the known locations of programs in a variable called \$PATH. Programs can be started in parallel using a '\&' sign, which can be very helpful when testing networked software. Modern shells such as Bash and Zsh provide autocompletion of possible commands when tab is pressed, just like modern code IDE's allowing you to auto complete code. Commands are always processed in the following structure COMMAND-NAME -ARG -ARG -ARG [FILE] [...] Some commands may take a fixed amount of arguments, while others maybe more variable, some may take in none. Most commands will perform some operations on a file. Almost every command can be run like so COMMAND-NAME -h or COMMAND-NAME --help to get a list of available options and how to use it. Arguments that are only 1 letter long will usually only use a single - and arguments that are words will usually use a double --. The shell has a current working directory, this is a folder that it is currently navigated too. When typing commands, file arguments can be given as a relative path to the current working directory. For example 'cp file1 file2' will copy file1 to file2, where both files are in the same directory. You can also use absolute paths which always start with '/'. There are 3 magic shell paths, ../ which is equivalent to the parent folder of the current working directory, ./ which is the current working directory and \url{~/} which is the currently logged in users home director (where their documents, downloads, videos, etc folders are stored). \subsubsection{The common commands} Items given in brackets are optional arguments \begin{description} \item{cp file1 file2} \\ cp will copy file 1 to the file 2, if file 2 already exists it is overwritten, if it doesn't exist created. The -r option is often used to allow it to co instead of individual files. \item{rm file1 [fileN]} \\ rm will delete any files provided, it is often used with the -r flag like cp is removed folders as well as files. The -f option is also used to avoid prompts of weather you want to delete the just deleting it straight away \item{mv file1 [fileN] file2} \\ mv moves 1 file to another location, it is equivalent to copying a file then deleting original. For some reason unlike rm and cp, it doesn't the -r option to work with folders. If file2 is afolder (something that ends in /) multiple files can be given and they will all copy to the folder. \item{cd [folder]} \\ cd will set the shell's current working directory to the folder specified. You can use this wi magic paths, for example cd .. will move you back one directory, and cd \url{~} will move you to you home directory. Running cd without any arguments is the same as cd \url{~} \item{ls [folder]} \\ ls will show the folders and files that are contained with in the folder provided. If no folder provide it will default to the current working directory can use the -l option to get file permissions and the -a show hidden files. (you can think of cd and ls equivalent clicking on a folder in file explorer and observing th contents). \item{cat file1 [fileN]} \\ Cat is a simple command that will print the contents o files given to it as arguments. \item{grep regex [file]} \\ Grep is one of the larger commands and has many abilities however its main one is to search through a file for a expression. If no file is given to the command, it will read from standard input (typing input) \item {sed expression [file]} \\ sed is a find and replace of a file (or stdin if no file is given), the expression should look something like this. "s/word/newword/". the 's' at the start tells you that it is in subsitution mode (other modes exist), the word is the expression to search for, and the newword is wat to replace it with. By default this will only substiute 1 word per line, if you want to do it more times the g (global) flag can be given like so, "s/word/newword/g". No matter how the file is give, via stdin or a file name, the output will be printed to stdout. \end{description} \subsubsection{Pipes and redirections} This is where the real magic Unix starts, as previously mentioned, Unix was designed with the id of making a small reusable tools, and this is how the tools can work together. One can take the output of one command pass it to another, or copy the outputs to a file. (A commands input is what is typed in by the user, and the commands output is what is printed to the screen) This can be done like so. ls > files.txt This will take the output of the 'ls' command and copy it to the file 'files.txt' (which is in the current working directory, we can tell this because it doesn't have any '/' in its name). You can also do: ls >> files.txt Which will do the same thing, however if the file already exists it will append to it instead of truncating it. This on its own is quite helpful, as many commands take files as arguments, and this will allow to use one command to help another (the '>' and '>>' is called a redirection in Unix). One can also use a '|' to take the output of one command and use it as the input to another. Here is an example: cat file.txt | grep [0-9]+ This has taken the output of the cat command, and put put it into the input of the grep command (as if you had typed what the cat command printed), and has append a special EOF character to the end of the cat command output, to specify the end of file. Grep has then read this input, and performs an operation on it, based on its arguments. \subsubsection{Shell scripting} This is where you take all the things you've used in the shell, and wr ite them into a file, as simple as it gets really. The only notable differences is the fact that you have to prefix a file with a special string to say it is a shell script and make it executable. This can be done like so: \begin{lstlisting}[language=bash, caption=example.sh] #!/bin/sh sed "s/word/newword/g" myfile > tmp myvar=$(grep "[0-9]+" tmp) echo $myvar \end{lstlisting} This simple example will run sed, and put its output in tmp, then run grep on the output and then echo the output of grep. The \$() syntax is used t o specify run the following value as a command and put its output here. A mixture of sed and grep can create some rather insane programs, in 2023 i use these two tools to do almost all of advent of code, by taking the inpu t and then manipulating into valid python code, which then solved the task for me . Sadly i dont have the scripts I used anymore, but there are some great videos of Russ Cox (co creator of the GO programming language) doing the same thing \url{https://www.youtube.com/watch?v=hmq6veCFo0Y}. \subsection{Device files} As mentioned prior, everything in unix is a file, this went all the wa y down to devices and hardware attached to the system. Here is a quick list of a ll the major files that you can use inside of programs to communicate with th e hardware on a device. \begin{description} \item{/dev/random (/dev/urandom)} \\ These are both random text generators that can be found on most unix systems. You can use commands like head (another small unix tool that prints the first 10 lines of a file), to read it like so \begin{lstlisting}[language=bash, caption=/dev/random example] key=\$(head /dev/random) \end{lstlisting} urandom works the same, although it uses a slower, more random algorithum to generate text. \item{/dev/null} \\ this is just a bottemless pit file, it contains nothing, it is often used to compare if something is empty, saying if the contents of a file is equal to /dev/null then its empty. It can also be used to delete files if you wanted, via mv file /dev/null (although rm does the same thing). \item{/dev/sdXN} \\ this is how devices are listed, your harddrive (or whatever the kernel detected first) is equal to /dev/sda and then each partion on the drive is equal to /dev/sda1, /dev/sda2, etc, etc. These file can be used for many things, for example if you want to burn an ISO or IMG file to a dvd/cd, the dvd will show up as /dev/sdb (or somethimes /dev/sr0 depending on weather you are using linux, bsd or something else), and you can use the dd command to copy it to the disk. You can also use the mount command to access the files on a drive. You can say mount /dev/sdb1 /mnt (which could be the first partion of a usb drive currently connected). This will mean that if you cd into /mnt, you will see the contents of the usb drive, and you can copy or edit files on /mnt to copy or edit the content of the drive. \end{description} \subsection{Some examples of things and how to try them} \section{The C wizard handbook} \subsection{Why is C the way it is?} } \end{document}