And a little bit of code reconstruction here and there.
Note: This is a follow up on my previous publication and is part 2 in an ongoing series on making a virtual filesystem in Go. The first part can be found here : https://itnext.io/a-virtual-filesystem-in-go-creating-our-foundation-9af62b0e82db.
Okay, that’s it. Let’s get started.
Over the past week the project’s had a little bit of editing done to the design of things. It turns out architecting and coding a project of this scale on the fly can lead to some unintended consequences. But it’s not like we never saw this coming. Things were bound to break eventually, that’s just what happens when you try to build a rocket as it rapidly approaches orbit.
This week we have some changes to make to existing pieces of code and also fresh new code that’ll breathe some life into our system’s soulless husk. We’ll be adding two relatively small features which will round off the user object for this version of the project and then we’ll talk about some of the refactors that ruined a significant chunk of the past few days for me.
We’ve got a bit to get through so get your text editors ready and let’s jump right in!
What We’ll Be Adding
The features in question will add some ease of usability to our shell and make our program actually runnable. They are:
- Command line history, autocomplete and cursor navigation
- An option to input our name on startup, so the prompt displays our name and not just a
$>next to our blinking cursor.
Once implemented, we’ll be able to startup the program; be requested to type in a desired username and then finally be spawned into our program with our chosen name as our prompt. We’ll then be able to autocomplete valid commands and have access to command line history.
It should be a relatively quick and easy config and by the end, we should have something that works like this:
Constructing our Startup
Let’s start off with a function that’s gonna help make these features happen as painlessly as possible; initUser() .
The code for init.go:
func initUser() *user {
username := setName()
currentUser := createUser(username)
return currentUser
}This function will be where all our startup initialization magic happens. For version 1, all it’ll be responsible for is setting a name for our user in the aptly named setName() function it calls first, which we’ll cover next (on a side note; any expansion that comes to the startup process like permission setting or logging in will happen here).
(also in init.go ):
package mainimport (
"fmt"
"log"
"github.com/chzyer/readline"
)// setName gets a custom username from the current user.
func setName() string { var username string
line, err := readline.New(">")
if err != nil {
log.Fatal(err)
} for {
fmt.Println("Please enter a username (1 for Anonymous):")
input, err := line.Readline()
if err != nil {
log.Fatal(err)
}
if input == "1" {
fmt.Println("Anonymous it is")
username = "Anon"
break
}
if len(input) > 2 {
fmt.Println("Welcome ", input)
username = input
break
}
}
return username
}
You’ll first see that we’ve imported a fresh new library straight from GitHub called readline . This lib will take center stage for the prompt as it gives us access to a lot of the shell features we want to implement.
Install readline with the following command:
go get -u github.com/chzyer/readlineFor this particular step however, the specific use of this library isn’t really consequential, we could’ve just used a bufio buffer instead and it wouldn’t have made a difference to it’s use case here. We just needed a way to input our username and nothing else.
I also added the option to type 1 for anonymous or guest users of the system. It’s a nice little detail to remind me that I have to go back and think about user security eventually. Not all viewers with read-only access want to have an extra password to add to the avalanche of passwords they already have to manage.
The step after setName is where we then create our user object with createUser. This function was extended with a new method. But before we get into that, let’s pause on implementation and talk about my design changes .
I didn’t know how else to segue into this topic naturally so I just opted to awkwardly break flow and put it here.
Sorry about that :/.
Thinking in Objects
Initially when I designed the filesystem, I wanted everything to be completely decoupled from one another. I thought of each piece of the project as an individual component, free from the influence of other parts; the filesystem is an object, the shell is another, the user’s a third. Sure they’d have to interact but it didn’t occur to me just how messy these interactions would be if they were all completely separate pieces.
Data and functionality has to be shared between these components constantly; the user’s permissions affect how certain library functions/shell utilities behave when a user, for example, attempts to edit a restricted file or access a restricted directory. The library being its own object would mean that each utility call (like open, mkdir or cd) would have the entire library passed into the filesystem object. It would quickly become an entangled mess and ruin the cleanliness my reasoning for keeping everything separate sought out to uphold in the first place.
It would basically look like C (another mistake I didn’t expect to make so soon).
And so this brings us back to our additions to createUser .
Readline is a collection of Unix-style command line helper functions that give us access to things like autocomplete, command line history and cursor movement for our shell prompt. It also has a variable where we can set the display name of the prompt. And so instead of making the prompt an element of the shell like it naturally felt it belonged, I opted to flex my underused OOP skills and integrate it directly into the user object instead and just hope that was the right thing to do. :D
Let’s take a look at initPrompt() and talk about it a little bit. This can be found in user.go
func (currentUser * user) initPrompt() (*readline.Instance) {
autoCompleter := readline.NewPrefixCompleter(
readline.PcItem("open"),
readline.PcItem("close"),
readline.PcItem("mkdir"),
readline.PcItem("cd"),
readline.PcItem("rmdir"),
readline.PcItem("rm"),
readline.PcItem("exit"),
)
prompt, err := readline.NewEx(&readline.Config{
Prompt: currentUser.username + "$>",
HistoryFile: "/tmp/readline.tmp",
AutoComplete: autoCompleter,
InterruptPrompt: "^C",
EOFPrompt: "exit",
})
if err != nil {
log.Fatal(err)
}
return prompt
}This made sense to me; the only variable that everything in initPrompt used was the current user’s username taken from the user object. And as this would eventually be changeable and, in turn, make this function reusable, I just decided to bundle everything together.
Setting the currentUser.username value to the Prompt variable would set our username as the shell’s prompt. Later on in execution, our shell would then make a call to user.initPrompt() to grab an instance of the prompt to use in its loop, complete with our name and command line functionality
As for the other variables, its not important that you know what they are for this project, but some trivia on shell stuff and the library won’t hurts so let’s go over those briefly. Feel free to skip to the next section if you’d like :).
The Config Options
HistoryFile is where our command line history info is stored. Every single time you hit ENTER to input a command, that command is written to a secret file (in this case, /tmp/readline.tmp ) that’ll be overwritten or deleted once you end your shell session and/or restart the program. See that convenient listing that happens when you press your UP or DOWN key and your previous commands get listed? That’s where the shell’s getting them from. It writes when you press ENTER on a non empty prompt, and reads when you press UP and DOWN to display what you’ve typed throughout your session. Pretty cool, no?
InterrupPrompt I’m not entirely sure what this variable is as I wasn’t able to get it working but I suspect it’s the value of the interrupt signal you could use to crash your shell like Ctrl+C. I’ll have to figure this out because what’s a shell without premature crashing.
EOFPrompt is the message that’s printed to your screen upon you exiting the prompt.
AutoComplete is where all the autocompletion code is setup. Simply go into the NewPrefixCompleter and add a new readline.PcItem value with whatever string you want and, like magic, that string you added is now part of your shell’s autocomplete list of strings. They can also be stacked on top of each other, so if you want to, for example, have an ls command which will then autocomplete it’s potential flags, you’d then set it up like so,
readline.PcItem("ls",
readline.PcItem("-a"),
readline.PcItem("-l"),
),
...ls will now autocomplete by pressing TAB, and a second TAB will cycle between autocompleting -a and -l . It’s quite intuitive really.
Okay, that’s all we need to think about with the readline library, if you’re interested in exploring it yourself, I’ll put a link to the repo at the end of the article.
Inspecting the Shell
You’ll notice that inside shell.go things are a lot leaner than last we left them. This will also be the project’s main entrypoint file from now on as the main is small enough to bundle it in.
Here’s our new shell.go:
package mainimport (
"strings"
)// shellLoop runs the main shell loop for the filesystem.
func shellLoop(currentUser *user) { filesystem := initFilesystem()
prompt := currentUser.initPrompt()
for {
input, _ := prompt.Readline()
input = strings.TrimSpace(input)
if len(input) == 0 {
continue
}
filesystem.execute(input)
}
}func main() {
currentUser := initUser() shellLoop(currentUser)
}
Continuing from after initializing our user object earlier, we’ll enter shellLoop() and our startup will come to an end with an initialization of our filesystem and the creation of our shell prompt.
You’ll also see that our long switch statement from our previous implementation is gone and has been replaced by a single call to filesystem.execute() .
Taking a peak into execute() , you’ll see the ghost of our old, outdated system haunting the confines of our filesystem object, again, turned into a method
(inside filesystem.go ):
// execute runs the commands passed into it.
func (fs * fileSystem) execute(command string) {switch command {
case "open":
fs.open()
case "close":
fs.close()
case "ls":
fs.listDir()
case "rm":
fs.removeFile()
fs.removeDir()
case "cd":
fs.chDir()
case "exit":
fs.tearDown()
os.Exit(1)
default:
fmt.Println(command, ": Command not found")
}
}
Also, I changed root to fs in the hopes of looking more cool and modern and stuff.
And with this change, we have officially both finished work on the user object and the shell, and begun work on the filesystem itself.
Now things get hard.
The Real Challenge Begins
I’ve been avoiding this step for as long as I possibly could, insisting on getting something we can start using done and dusted as soon as possible.
True, what we have isn’t really impressive, but it’s something built and that works that we can use. Use meaning that we can press buttons and things happen on screen, albeit not very “useful”.
Admittedly I’ve had a lot of trepidation about this step, mainly because a filesystem in general is both a hard thing to make and can get quite large in scope. This is also compounded by the fact that I’m both planning and writing this project on the fly. So to mitigate potential future damages, I’ve decided to follow a few conventions established by other filesystem Go projects; hence the dropping of lib.go entirely and incorporating it’s code into the filesystem directly.
This will both help with eventually extending the filesystem with helper functions and interfaces from other libraries and will also make debugging easier to work with with a common design.
But now that I’ve taken a bit of time to think about these things, I finally have a roadmap of what needs to happen next. It’s made easier now that everything that’s left is in the filesystem part of things.
Wrapping things up — Onwards to part 3
Our task list of what’s left for V1 looks like this:
- Setup
initFilesystem()andtearDown() - Build the filesystem’s library utilities (open, close, etc.).
- Handle filesystem navigation.
- Setup a load/save state system.
It doesn’t look like a lot but there’s a massive devil in each of these details. The first item is possibly the last “easy” thing left to do. It mainly requires knowledge of basic recursive directory traversal. Then directory navigation would naturally come next, then the utility functions and lastly, controlling saving and loading of states.
Version 1 has no security whatsoever. Anyone can run it however they please. Assuming it all goes well, it’ll be a fun challenge to think about that later on.
And on that note; the real battle has just begun!
As always; here’s a link to this week’s updates: https://github.com/AlysonBee/GoVirtualFilesystem/tree/master/src/02
Here’s the link to the readline library: https://github.com/chzyer/readline
Again, I’ll be polishing this week’s README.md and updating the unit tests to accommodate our changes.
Alright, till next time!
