Go Virtual Filesystem — And a Shell Too, I Guess

Yup, that’s what we’ll be talking about this week.

Its funny because it’s a shell… :|

I love how I always commit to completing or touching on a certain component on an article and the end up having to do something different entirely because it turns out that working on that component requires endless groundwork I never consider until its too late. Yes reader it happened again; we’ll in fact not be further building initFilesystem this week like I said we would (good thing I didn’t promise anything :D) and here’s why.

A Few Days Ago…

Like a good, honest and hardworking engineer, I did the right thing by starting off with the tests I’d use to test out initFilesystem before proceeding to write it out. These included functions I’d use to be able to tell that our directories and files were created and created in the right order. The first approach of just, say, writing a directory traversal program that would go through my entire filesystem and print everything in the order it finds it (depth first search style) seemed like a good idea and an easy thing to do. But this is when I ran into another new nuance about Go (or most probably OOP in general) that I didn’t understand.

A Note on the Filesystem’s Structure

An overview of the filesystem’s structure. I hope this makes sense :’(.

Each filesystem object is a struct that represents a single directory. Each directory contains files and subdirectories. Files have their own individual struct that are stored as an array of file structs. The subdirectories are, in turn, represented as an array of directory structs. Each directory you switch into will hold a list of files and directories that it holds in their respective struct arrays. Here’s where the problem with my idea of keeping every utility, specifically cd as part of the filesystem object.

So cd both being called as a method of the filesystem object while simultaneously expecting to change the object that’s calling it is a violation of scoping rules. The problem we have is that the filesystem object when it exists inside chDir can only exist within the scope of this function and any changes to it would be lost when it inevitably returns.

A short example:

func (fs *fileSystem) chDir(dirName string) {
   ...
   // this updated 'fs' exists only inside chDir's scope. 
   fs = newDirectory(dirName)
}
...

// this 'fs.chDir("./hello/world")' and the 'fs = newDirectory(dirName)' aren't the same thing. A change to the latter won't affect the former.
fs.chDir("./hello/world") 

So keeping chDir part of the filesystem just felt like an untidy idea; I didn’t want the fs object calling it to also return a new fs instance that overwrites itself. My intuition was also affirmed by the fact that a quick read on Wikipedia about how traditional shells handle themselves have the cd functionality directly built into them. So sticking to a convention is always safest; chDir was taken out of the fileSystem object and was moved to the newly created shell object which we’ll talk about in a little bit.

Giving the Shell A Little More Spotlight

So it turns out that to make this filesystem as intuitive as possible, the shell needs a few special features of its own that would turn it into its own separate entity, rather than just a convenient UI tacked onto the filesystem. This means that a few critical features and tasks for navigation belong exclusively to the shell; one of those important features being the ability to change directories. More might follow but lets keep it simple for now.

A Little In Depth in Directory Changing

Bringing the shell to the front and center stage required that I give it as careful a consideration as I did the filesystem. It’s no longer just a piece of the filesystem that could be hacked away if needed, it now needs its own helper functions and structures. So the best place to start I think is to focus on a helper function that chDir will depend heavily on as well as any other command we implement that takes a valid path as an argument.

- chDir

Let’s take a look at chDir now that it’s been moved into a brand new object; shell ;

code in shell.go

// chDir switches you to a different active directory.
func (s * shell) chDir(dirName string, fs *fileSystem) *fileSystem {
if dirName == "/" {
  return root
 }
return s.verifyPath(dirName, fs)
}

It’s pretty simple enough. We’ve made it a method of the shell itself and it now takes in a fileSystem object and returns one too. The first check is a direct navigation to root check. If the only argument to cd is a slash, chDir returns the root value that we originally set as the root node upon calling initFileystem . Remember ;

filesystem.go

func initFilesystem() *fileSystem{
   root = &fileSystem{
     ...
   }
   fs := root
   return fs
}

That’s the root value we’re referring to. It’s a global variable so it’s accessible from everywhere.

Anyway, returning to chDir , let’s take a look at the following function. It’s that helper function that’ll help us with verifying the validity of directories;

shell.go

// verifyPath ensures that the path in dirName exists.
func (s * shell) verifyPath(dirName string, fs *fileSystem) *fileSystem {
 checker := s.handleRootNav(dirName, fs)
 segments := strings.Split(dirName, "/")
 for _, segment := range segments {
  if segment == ".." {
   if checker.prev == nil {
    continue 
   }
   checker = checker.prev
  } else if s.doesDirExist(segment, checker) == true {
   checker = checker.directories[segment]
  } else {
   fmt.Println("Doesn't exist")
   fmt.Printf("Error : %s doesn't exist", dirName)
   return fs
  }
 }
 return checker 
}

There’s a lot going on in the code above, so let’s unpack a little bit of it. The gist of what the code does is that it loops through the segments of the path argument you pass in, for example cd directoryA . This function splits the string by the / character that denotes a directory to determine if directoryA exists (turning directoryA into an array with only one value if no / are found).

Ah, but it looks like quite a lot just for a simple directory check, no? Well, say you decide to pass in a ../../../directoryA/../directoryb/../../../../../test1 solely because you want to break my project, it would split this string into quite the large array. The .. are to traverse a directory upwards, and a named string is to traverse into that specified directory name at that level if it exists. On the first occurrence of a nonexistent directory, the function fails and exits and we’re kept to the directory we’re already in. But if the path is valid; we’ll be navigated to it. Let’s look at a few technical details in the code that’re worth mentioning;

checker.prev . You’ll remember that if we look at the fileSystem object, it has a prev pointer value that points to its parent directory (or nil if we’re at our root). This value’s used to traverse backwards to with each .. it comes across.

The next is this curious line; checker = checker.directories[segments] . This detail’s a bit of an upgrade on the old fileSystem struct;

filesystem.go

Before:

type fileSystem struct {
 name        string              
 files       []file  
 directories []fileSystem
 prev        *fileSystem
}

After:

type fileSystem struct {
 name        string     
 rootPath    string // more on this later.      
 files       []file  
 directories map[string]*fileSystem
 prev        *fileSystem

The purpose of the directories variable doesn’t change, but making it into a map from an array makes accessing directories a lot faster as map access is always takes a single step. An array on the other hand, in a worst case scenario, would take as many steps as there are directories at that level; this would require many loops for directory accesses and checks.

Lastly, there’s the shell method doesDirExist .

shell.go

// doesDirExist checks if the dirName directory exists.
func (s *shell) doesDirExist(dirName string, fs *fileSystem) bool {
 fmt.Println("DoesExist entered")
 if _, found := fs.directories[dirName]; found {
  return true
 }
 return false
}

This is where the benefit of having single step access time into our directory list comes handy. Running chDir on a deep path could take time if it had to run a for loop with each directory we come across in the chain.

Note: so I got this specific way of checking if a key exists as show above from a specific Stack Overflow response so admittedly I don’t know why that works. It’s the first time seeing convention like that in an if statement so take that with a grain of salt. Incidentally let me know if you know why that looks the way it is, especially that ; found { bit.

- mkDir

The mkDir function isn’t all that different to initFilesystem . It simply initializes a new filesystem object and adds it to the array. Error checking still needs to be added to ensure a directory with the same name doesn’t already exist (and indeed error checking is missing in a lot of places), but it’s simple enough to understand;

filesystem.go

func (fs  * fileSystem) mkDir(dirName string) error {
  newDir := &fileSystem{
  name: dirName,
  rootPath: fs.rootPath + "/" + dirName,
  directories: make(map[string]*fileSystem),
  prev: fs,
 }
 fs.directories[dirName] = newDir
 return nil
}

The rootPath is the only addition added to the filesystem. This will come in handy when creating the pwd function which we’ll cover briefly in the next section.

A compelling reason to eventually move mkDir to the shell object is that it also technically uses a path as its argument which would have to be extensively verified; same goes with listDir . In fact, in the end, our filesystem might not have any user related functions in it at all because paths are so important in making a stable shell.

But this article’s getting pretty long so we’ll save those mods for next week.

Extra Additions and Honorable Mentions

A few pieces of code to mention that weren’t very technical to make but are changes I added to expand the filesystem are as follows:

• ListDir — A discount ls that just lists our files and directories. It’s mainly used to help check to see if our filesystem works for now.

filesystem.go

func (fs  * fileSystem) listDir() {
if fs.files != nil {
  fmt.Println("File:")
  for _, file := range fs.files {
   fmt.Printf("\t%s\n", file.name)
  }
 }
 if len(fs.directories) > 0 {
  fmt.Println("Directories:")
  for dirName := range fs.directories {
   fmt.Printf("\t%s\n", dirName)
  }
 }
}

Basically it loops through the file list and object list; if it’s there, print it. Files haven’t been defined so we’ll ignore that for now.

• pwd — The print working directory command. This one is assisted by a little edit I made to the fileSystem object. I added a variable to it called rootPath that you’ll see in the struct definition above. mkDir was edited to accommodate the addition of the root path to this object.

Now the root is generated from the current path we’re on with the newly created file tacked onto it. And so pwd() is a simple printout of this value at the level you’re on.

filesystem.go

func (fs * fileSystem) pwd() {
 fmt.Println(fs.rootPath)
}

• clear screen — the option to clear your screen when you run the clear command. Not a lot of thought went into this one, all I do is just run the clear (or cls for Windows users) binary in the background using the exec.Command() feature.

shell.go

func (s *shell) clearScreen() {
 clear := make(map[string]func())
 clear["linux"] = func() {
  cmd := exec.Command("clear")
  cmd.Stdout = os.Stdout
  cmd.Run()
 }
 clear["windows"] = func () {
  cmd := exec.Command("cmd", "/c", "cls")
  cmd.Stdout = os.Stdout
  cmd.Run()
 }
 cls, ok := clear[runtime.GOOS]
 if ok {
  cls()
 }
}

• initShell — shell initialization was added. The env variable that it will have was originally going to be used to hold the PWD but was dropped in favor of just printing out the rootPath . I like this method most because it prioritizes time access over space/storage as making something that edits the PWD env variable would have a lot of looping and string editing to work with and I enjoyed Doctor Strange more than the first Captain America movie (secret reference Easter egg). It’s just faster to print rootPath .

shell.go

func initShell() *shell {
 env = make(map[string]string)
 return &shell{
  env: env,
 }
}

• Shell loop additions — I edited shell loop a bit, inspired by the C shell I wrote during my studies. The command is split so the commands can be checked separate to their parameters right away. All calls to shell commands are checked first before passed into the filesystem object for further checking.

main.go

// shellLoop runs the main shell loop for the filesystem.
func shellLoop(currentUser *user) {
 shell := initShell() // iniitalize shell commands
 fs := initFilesystem()
 prompt := currentUser.initPrompt()
 for {
  input, _ := prompt.Readline()
  input = strings.TrimSpace(input)
  if len(input) == 0 {
   continue 
  }
  comms := strings.Split(input, " ") // they're split beforehand
  if comms[0] == "cd" {
   fs = shell.chDir(comms[1], fs)
  } else if comms[0] == "clear" {
   shell.clearScreen()
  } else {
   fs.execute(comms)
  }
 }
}

And at the end of this week, our shell can now do this:

I’m working on making them clearer, I promise :’(

Next Step — Onward to Part 4

This was one of my longest articles thus far because of the chaotic nature of editing and reediting multiple components. For the next article I kind of want to get back into the flow of targeting something very specific. Things make a lot more sense now that the shell object has been introduced and I got to experiment with actually tampering with file creation.

So, back on track, changing directories is handled, making directories is handled. I’ll probably be spending some time on polishing what I have so far. The cd command doesn’t detect ../../ as a valid path which I recently discovered (a day before publication). So I’ll be testing a bit which might push back new updates and next week’s article drop.

Next would be to work on file creation, file and folder deletion and then to finally implement initFilesystem . It’s quite the elusive task.

The link to this week’s progress: https://github.com/AlysonBee/GoVirtualFilesystem/tree/master/src/03

Unit testing (which I’m falling behind on by the way) and the README.md to follow.

Till next time.