I find that writing projects makes me learn a lot more than just reading about writing projects. This is, of course, fairly self-evident, but I think the vast amounts of things I know I don’t know often lead me into rabbit holes and trying to prepare to do a better job - instead of just actually doing it. As they say, perfection is the enemy of good.
To combat this, Swostik and I decided to have weekly projects: small programs we work on together with short deadlines, generally about a week. The idea is to have something to show by the end of each “sprint”, to borrow Agile terminology. Primarily the hope is that doing this means we will actually finish our projects, actually practice writing code, and, importantly, avoid the sprawl of undefined endings.
books is our first “sprint” project. The idea is simple: a program that can store data about a personal book collection. Our requirements were that it should include basic book-data, such as title, author, ISBN, etc as well as a start-date and finish-date parameter to keep track of books we’ve read. Additionally, I wanted to make sure we only used the Python standard library1.
Planning
For this application, I figured the first step was to define how a user will interact with it. This becomes a flexible concept down the line, but having a starting set of actions to describe what the tool will do seemed rather useful. The result was a set of CLI commands that described what we wanted:
# Add a book
books add --title "Dune" --author "Frank Herbert" --format print
# List books
books list
# Mark a book as finished using short flags
books edit -t "Dune" -fd 2024-06-01
# Delete a book
books delete -t "Dune"This approach is largely informed by the procedures and ideas described in How to Design Programs. It is also, I think, a good (although imperfect) example of creating OOP interfaces, if not formally. The example CLI commands above gave us a map to work on - a contract, if you will - that we must fulfil in order for the program to function.
Aside
This particular point becomes relevant later. I am a big fan of this idea of interface design, since it helps me abstract away the specific implementation. It is a very useful way to ensure that SOLID is being applied, too!
With this in mind, the next step we took was to come up with a system architecture. To avoid code sprawl and unmaintainable garbage, we elected to follow the tried and true Model-View-Controller architecture (even though we did not specifically call it that). My instinct was to borrow from Domain Driven Design and also narrowly define the fundamental data model we wanted to work with, which is what became the Book model. In the end, the design consists of four parts:
- The
Bookmodel; this is the fundamental data that the entire program uses. - The
Libraryclass, which represents the actual application logic. It is what might be described as the core part of the program. - The
Repositoryinterface, used to collect data into one place2 - The
UI(which for a week long project is simplyapp.py).
Building
Because of how we defined the different components of the program, the going theory was that in general we should be able to develop each piece independently if we agreed to a few simple facts. The UI as described above set the contract for how a user might interact with the Library, which means the connection between those layers was more or less defined. To formalise the next step, then, we wrote an “interface” for the repository:
class Repository(ABC): # ABC is an Abstract Base Class; it means this specific class cannot be instantiated, but it can be used as a superclass
@abstractmethod
def __init__(self) -> None:
pass
@abstractmethod
def save(self, book: Book) -> None:
pass
@abstractmethod
def list(self, **identifiers) -> List[Book]:
pass
@abstractmethod
def update(self, book: Book) -> None:
pass
@abstractmethod
def delete(self, book: Book) -> None:
passThis meant that when I went to write the Library logic, I simply assumed that there would be a Repository with those methods available. The actual work of creating the Library was first to make sure it worked3 and then, given that it can interact with the Repository, that it can enforce rules if applicable. Note that in writing this file, I completely ignored the existence of a UI layer.
The fundamental concept that makes all of this work is composition; the Library is composed with an abstraction that uses the Repository interface. From this perspective, I do not care how a specific repository implementation will list the data; I just know that it will happen and that there is a list method, for example, I can call to list items.
class Library:
def __init__(self, repository: Repository):
self.repository = repository
def list(...):
...
self.repository.list(...)
...
# Rest of codeBeyond other details, a big factor that helps all of this work is the Book model I mentioned before. If you look at the Repository base class above, you’ll notice that aside from the list() function, everything is designed to accept a Book as a parameter. This constraint meant that the Library always has to construct a valid Book before sending it to the repository layer - which means all error handling, etc. must happen at the library layer. The separation of responsibility also simplifies the construction of any specific repository: a book will always be given to it.
Aside
I am learning Java and I’m no expert, but I think this is akin to a Data Transfer Object (DTO).
With both the core Library in place and a Repository (or the specific implementation we did, SQLRepository), doing the actual user-interface was a lot more enjoyable. We could focus entirely on how to parse arguments and how to create commands rather than focusing on the actual logic of data manipulation and storage (aside, of course, from making sure we have all the inputs required to create a book).
We started by defining the user interface actions we wanted, then we designed the other parts of the program (in a way such that there is no dependency on the UI itself). An advantage of this is that our UI can be trivially swapped for something else: instead of a TUI, perhaps we want a webserver, and all we have to do is create URL endpoints that call the Library in the exact same was as the CLI. To be specific, consider the following part of our app.py:
def main():
loadscreen.welcome()
print("Welcome to Books ")
print("For help enter : books -h")
parser = cli_parser()
while True:
try:
user_input = input("> ").strip()
if user_input in ["exit", "quit"]:
print("\nThanks for using books")
break
if not user_input:
continue
args = parser.parse_args(shlex.split(user_input))
command = {
"add": add,
"list": list_,
"edit": edit,
"finish": finish,
"delete": delete,
"import": import_,
"export": export_,
"recap": recap,
"clear": loadscreen.welcome(),
}
func = command.get(args.action)
# print(vars(args))
if func:
func(vars(args))
except SystemExit:
continue
except KeyboardInterrupt:
print("\nThanks for using books")
break
if __name__ == "__main__":
repository = SQLRepository()
library = Library(repository)
main()This is the fundamental part of the UI: an endless loop that parses arguments from the command-line and routes them to specific functions. However, the repository and the library are instantiated outside of this loop; they could easily be endpoints for a different version of main() that defines URIs instead.
Learning
This was a fun project to do. The main thing wasn’t so much the specifics of what the program does (in fact, I have a different, much larger project that implements a similar idea), but to practice different parts of the development cycle. Although I was already familiar4 with SOLID and Domain Driven Design, applying it into a simple program and being able to focus on those parts was great.
In particular, I found it rather exhilarating when I was able to modify and change individual parts of a system without cascading problems everywhere. I recall years ago I wrote a smart-light automation script with a horrible, horrible set of functions that combined all layers of code into a single, unified, abominable mess. Seeing it now makes me cringe, in the same way reading my shitty poetry from when I was en edgy 15-year old5.
Perhaps the most useful (to me) thing I learned, though, was being able to start, develop, and finish a program while working with someone else. A friend of mine recently told me that “solo projects are a tyranny”6, and nothing helps you write more readable code than the explicit expectation that someone else will actually read it7. Having to establish a git branching strategy, having trust that my friend will write system components that will work with that I write (via our mutually agreed upon interfaces) was definitely a lot more interesting than just writing stuff on my own.
Footnotes
-
Frameworks are great and all, but I think their unconsidered use leads to a few problems; you spend more time learning the framework rather than your idea, you have to deal with a prescribed way of doing things, you have to manage dependencies and their security, and they easily lead to sprawling code and feature creep. ↩
-
Note that I did not say persist, although that is the more common use for it. But a repository interface can also be a repository that lives in-memory, so I suspect it is technically incorrect to refer to this as a persistence layer. ↩
-
This seems obvious, but I know from experience that it is easy to jump into all the business rules and fancy logic only to find that the code is fundamentally flawed. ↩
-
Read: I found a couple of articles ↩
-
I am told this is a sign of growth, and sure hope that is the case. ↩
-
Incidentally, this is the same friend for whom I wrote the light automation script. It’s a testament to our friendship that even after he used it we remained friends. ↩
-
I suppose this too is true of writing anything, like this blog. ↩