Lessons Learned from Building NCP Registration

View post footer for the background on this website.

It has been over a year since my course registration application, informally known as NCP Registration, was launched to the students of Northside College Prep. Development started in early July 2012, and continued though the launch in September 2012, feature additions ceasing sometime in Spring 2013. Upon graduation in June 2013, I was no longer responsible for the application.

Features #

The application performed the following:

It should have also performed automatic course assignment for those who did not sign up for a course, but that feature did not reach production-ready status.

Technology Stack #

Powering the application was this technology stack:

What Went Wrong #

In building this application, I ran into many challenges, some of which were easy to remediate and some were more difficult. In this first part, I am going to talk about some of the things that went wrong.

Extremely long load times #

As registration for courses started at 7AM, students were eager to select classes. Over a hundred site visitors at one time meant for very poor performance without the appropriate hardware and caching mechanisms in place. With the database buckling under the load on the first day, there needed to be fixes, and fast. Just like HealthCare.gov…

Luckily, after some quick fixes, this issue was able to be resolved somewhat within the first two days.

Unreliable registration statuses #

Due to the server issues, some requests were timing out before necessary actions could be completed. This caused a chain-reaction which led to some users thinking they were registered for a course when they were not.

Changing the frontend code solved this problem, but it did not revert the misconceptions of many users.

Lack of communication #

Any development project suffers shortcomings when information is not properly communicated to players involved. From not fully understanding the assignment when it was initially given, to not communicating the issues to users in time, to administrators not fully understanding how to use the application, the effectiveness was diminished.

These issues were slowly resolved over the months the application was developed, and the amount of communication had greatly increased by the time second semester rolled around.

Usability & design flaws #

Design is very important in any application, and the usability/user experience determines whether users will make complaints. In this case, the design appeared fairly solid to students, but for teachers and administrators, the excessive links and options proved cumbersome. This was one of the few issues I could not resolve completely, so instead a manual was created and new shortcuts were made.

Overly complex implementation method #

While a modular system such as Drupal may have been good-intentioned in its use, many of the problems faced could have been avoided if a different approach was taken. This approach would focus more on the problem at hand, and not take into account the what-if’s as much. This complex implementation was of creating a system that handled attendance and registration together, patched together through Drupal modules not truly designed for their new use.

Lack of solid testing #

No application can be considered successful, much less working, if it does not pass all the tests of a production environment. One of the issues that I faced as a young developer was not knowing how or having the time to implement good testing protocol into the application, which would detect edge cases as well as my manual tests. One cannot have innocence forever.

What I Learned #

Through those failures I have mentioned, I was able to get a clear understanding of how to handle projects like this in the future.

Shiny black boxes aren’t always better #

Having used Drupal for two other projects in the past and seeing how easy it was to create new types of content, I viewed this project from a content management system context. However, when building the application on top of Drupal, I ran into problems that would have been avoided if I had coded everything myself. For example, Drupal’s core contained a lot of “fluff” that wasn’t necessary for building the site, and so this “fluff” slowed the rest of the site down. If I were building this again, I would weigh the trade-offs of using Drupal versus another framework versus a custom-built solution more seriously.

Less complexity, less failure #

When building web applications for clients in government and education (where large bureaucracies are involved) there are often changing requirements. However, the squabbles that result in changed requirements are often over minor details rather than large conceptual changes. Therefore, if the code you build is simple, where simple is synonymous with highly abstracted, then it becomes easier to adapt to change. In this case, I did not know the details of what I was building until a month into the project. However, if I had built the system to revolve around signing up more generally for a course rather than a colloquium or an X/Y block, then when changes needed to be made (e.g. to account for special periods during finals) the system could adapt better. As famous computer scientist Edsger Dijkstra once said, “Simplicity is prerequisite for reliability.”

Test before you ship #

One of the biggest failures of IT project rollouts is when the software is not tested before it is released to the public. We have seen this in the last few months as both HealthCare.gov (the federal healthcare exchange) as well as Ventra (the CTA’s new payment system) have had massive problems due to a lack of testing. This was also the case in my project, for load testing was not performed adequately and some code that was pushed live did not run as expected. To solve this problem, software engineers use tools like unit tests and development servers to avoid using real users as guinea pigs for their hole-ridden code. I now use a development server for my projects and will be learning unit testing soon.

Avoid bikeshedding #

As developers, we often feel the need to argue over what programming language or framework to use, and other trivial components. We may not agree with the way the business wants their product to be built, but not always do we have the privilege to make the high-level decisions. Therefore, while it may feel pleasing to think about a problem from a computer scientist standpoint, it can also waste time from accomplishing the real task at hand. My philosophy is now that you should get the task done the first time using what you know best, and then focus on optimization/picking the best tool for the task, regardless of experience with it.

Do not alienate your users #

Us tech people get it. Problems do happen, projects get postponed, bugs occur. As much as dealing with customers/users may be unpleasant, it is our job to communicate any issues to them. It is always better for someone to know that their house is burning than to try to put out the fire before they wake up. The more they know, the more understanding they can be (this is usually the case, but not always). Bottom line: more communication equals more success.

Ask for help #

On the point of communication, it is very important to know when you’re no longer able to complete the task on your own. Even more importantly is the ability to summon the courage to ask someone for assistance. One of the biggest lessons I’ve learned from this is that being a sole developer does not go very well. As I’ve heard countless times before, humans naturally work better in groups, and for good reason. In software development, the ability to distribute the work and check on each other is key. Having a partner in a project can be very beneficial. I have learned that I need to first know when it becomes too much, and also know how others’ can help me when I ask them. Asking for help may not be enough - show someone how they can help. And if you are ever asked for help, pay it forward.

Presently #

Today, I am working on new projects which make use of the experience I have gained from this project. Later today, I will be launching the new version of YSFlight Headquarters, 4.0, which myself and a few others have been working on for the last few months. The registration site is no longer in use, but the replacement has had some recent development. You can view progress on it here. While I am still not certain if I want to become a web developer or something else, I know that my love for computing will be around forever. This project, and the aftermath of it, has only made my passion stronger.

If you enjoyed this post, I would be honored if you would follow me on Twitter. #

Background: NCP Registration was the web application that students at Northside College Preparatory High School (NCPHS) used to register for semesterly and weekly courses. It contained “colloquium” courses as well as “X/Y” courses. Both of these occur on Wednesdays at Northside, on regular 5-day weeks (M-F). The application can still be accessed here. I was hired by the school (as a student worker) to build this application. The following are my thoughts only, and do not reflect the views of NCPHS, Chicago Public Schools, or any other entity.

 
2
Kudos
 
2
Kudos

Now read this

Status Report

What projects is Eric working on, and what is the status of each? Find out here. Will list in order of importance. Packback # Takes up about 75% of my time most days. Enjoying my time here, feeling more and more like a real software... Continue →