System Overlord

A blog about security engineering, research, and general hacking.

Things I Wish Undergrad Had Taught Me

This is not an attempt to knock any particular program, professor, or course of study.  It's just some things that I think should be included in an undergrad CS program that I don't feel like I got.

  1. Serious study of data structures and algorithms.  While I know how to implement a linked list, structs, classes, vectors, and other data structures, not a whole lot was said about the best use cases for each.  That's something I've had to discover on my own.  And the most complex tree we discussed was the Binary tree.  We never talked about balanced binary trees, red-black trees, or generic n-ary trees.  Although I was taught the general idea behind Djikstra's algorithm, and can tell you the big-O runtime for about a half-dozen sorts, practice implementing them and discussion of their comparative strengths and weaknesses is not something I remember from my undergrad.  Also, there was NO discussion of time-memory tradeoffs involved in implementing some of the algorithms.  In fact, (and I'm embarassed to admit this) I only recently found out about the in-place implementation of quicksort!
  2. How to find your focus.  If there's ever been a real-world example of an NP-complete problem, it's finding your niche.  I'm still searching, and as I get into more things, I'm finding more interests than I am able to exclude.  IT/Computers/Technology is a massive field and even narrowing it a little is hard.  About the only things I've narrowed down are that I don't want to do end-user support, that I don't want to manage people, and that I want to work with/develop Open Source.  Oh, and that I like not doing the same thing every day.  (As it is, my current job is getting on the monotonous end of things.)  I hope I'll find my focus before its too late.
  3. How to develop with others.  This is a skill I've developed over the past few years of the "real world", but I'm not sure everyone I've worked with has gotten it down.  There were too few group projects in my undergrad, and those that I had were comparatively small.  We never had the big software engineering problems, and never really had to develop good APIs for others to depend on.  The division of work never seemed to be "you do the UI, I'll do the database components, and he'll do the business logic."  It was always "you do the UML diagram, I'll do all the code, and you do the final report."  That's not how it works in the real world (ok, well, sometimes it is, but it's not how its supposed to work).
  4. How to effectively use source code management.  Using SCM is critical to any serious development.  Not once in my entire undergrad career was that discussed.  No mention of any SCM.  While my experience in open source had led to me using and understanding SCM, I can say that I've seen how well prepared others are to use it -- and it's pretty scary.
  5. How to do requirements definitions and other software engineering tasks.  When I started my undergrad, there weren't really any dedicated software engineering programs -- everyone did CS or IS.  In the CS side of things, there was one software engineering class.  You can't learn to estimate time, do requirements definitions, manage deliverables, and all the other tasks that go into a software lifecycle in one class.  While I realize not every CS student will end up doing software engineering, the software engineering class should be early in the program (in my program, it was nearly at the end) and those concepts should be incorporated into every major project you do for the rest of your degree.  You've gotta do things more than once to really understand it.
  6. How to do dev/test/prod.  Much like #5, the words "unit testing" never came up in my undergrad program.  There also wasn't really any discussion of maintaining existing software, and of the different environments.  I knew about them, but not from my undergrad, and I've had to learn a lot about them "on my feet."  I'm still trying to get some of our practices at my job into this lifecycle in a sane manner, but it turns out: doing things the right way requires more work up front.  It'll save you in the long run, but it's hard to get that initial investment when it looks cheaper to "fix things later."  (It's not, by the way.  Doing it right the first time is always cheaper.)

I'm still learning a lot -- but if you're not learning, you're stagnant.  There are just some things that make you slap your forehead when you realize how nice it would have been to know those skills 5 years ago.

A Hard Lesson Learned

For a few months now, I've been working on a side project for a local girl's volleyball club.  While the people I'm working with are very nice, this whole project has been a lesson in how bad of a businessman/project manager I am.  I'm struggling with whether this is a sign I should stop taking on these side projects, or if its a sign that I really need to pay more attention to the business side of things.  If nothing else, I hope this will serve as a warning to others on what not to do.

Lesson One: It's Going to Take Longer Than You Think

No matter how much time you think it's going to take, you're likely underestimating it.  When I first got involved in this project, I expected it to be about 100 hours of work.  I stopped counting the actual hours invested on this project at 200 because it just got to be too much to think about.  There are components to this project that are similar to, but not exactly the same as, tasks I had completed before.  However, these tasks turned out to be far more involved than I had anticipated, and so my initial estimates were quite a bit off.

Lesson Two: Have a Detailed Statement of Work

At some point, there's going to be a difference in interpretation of some part of the project design.  No specification can be so complete as to eliminate every possible point of contention.  However, the more detailed the original statement of work is, the smaller these issues should be.  Having a very detailed statement of work allows both parties involved to have a clear understanding of what is expected, and prevents the last minute "it was supposed to..." statements.

Lesson Three: Have a Contract

I'm not saying you need some behemoth document that requires a $150/hr lawyer to understand, but there needs to be a legally binding document that includes when you will deliver the work, in what format, how much you will be paid, and when you will receive payments.  Having this makes sure there's no surprises for either party involved -- you know your deadlines and when and how much you will be paid.  The customer knows when they will receive what they're paying for and how much that is.

Lesson Four: Have a Change Request Process

There will be changes during the development or delivery process.  It's going to happen, and you can't change that.  What you can have is a process to deal with it.  The customer needs to know what costs they're incurring by asking for changes and whether or not the changes will affect the delivery date.  You need to build a revised detailed spec with the new changes in it.

The Bottom Line

I hate paperwork, meetings, and managing.  However, it turns out that, to an extent, those may fit in the category of "necessary evils."  If you read through the list of lessons above, you can probably imagine the mistakes I feel I've made on this project.  I've unfortunately lost quite a bit to this project, including most of my weekends for the past 3 months and a LOT of sleep.  Hopefully, I'll learn how to do things the right way -- and maybe you won't have to learn this lesson the hard way!

What I learned from Steve Jobs

Unless you've just awoken from a coma, you're probably well aware that Steve Jobs passed away a few hours ago.  It might be the very first time that the death of a "celebrity" has saddned me.  Steve was more than a celebrity, he was visionary like none other.

Steve had a vision that was unmatched by anyone else, even his Apple cofounder Steve Wozniak.  The Woz and I are much more on the same wavelength -- fascinated by the technology, fascinated by doing things just to see them done.  Jobs, on the other hand, saw the bigger picture instantly.  He saw how the technology would change the world, and he got there first (most of the time).  Lesson one: See the big picture.  Even if you don't control the big picture, see how your part fits into the big picture, and make it better.

Steve had many successes, but he wasn't infallible.  He worked a number of menial jobs in college, and got fired from several of them.  The Apple Lisa left a lot to be desired, both in technology and in sales.  He was fired from the company he cofounded.  Did he give up, or decide he shouldn't be in the business?  No.  Steve picked himself up, and showed people that they were wrong about him.  Lesson two: Fail.  Learn from failure.  Repeat until you succeed.  (A corollary lesson: being told you can't do something should make you want to do it even more.)

The biggest thing I've learned from the life of Steve Jobs came about just tonight.  No matter how rich you are, how popular you are, how smart you are, no matter what you've done in your life, it comes to an end entirely too soon.  Steve didn't wait for things to come to him, he made things happen for himself.  Time and time again, he did thingsLesson three: We all die, and for many it's entirely too soon.  So get out there and do things.  Make your obituary interesting.

RIP, Steve.  You were a pillar of the technology industry.  You will be missed.

Coming Drupal Trends

Based on Drupalcon last March and Drupalcamp Atlanta this weekend, I've seen some growing trends in Drupal.  While some of them might "already be here" I don't think everyone's doing them yet.  Some of them apply to web development in general, while others are more specific to Drupal.

Adaptive Web Design

We all know mobile is here and is going to stay.  However, the days of 23-30 inch monitors aren't over.  Making something that is highly usable on both ends requires adapting to the user's platform (hence adaptive design).  Themes like Omega, AdaptiveTheme, and their derivities are probably going to replace base themes like Zen in order to make things more "adaptive."  It's worth noting that Zen can be adaptive with media queries, but it's not designed for it from the ground up.


I'm sure this will come as a great surprise.  HTML5 is here to stay.  Which, if we're at all lucky, means that Flash will be reserved for those rare cases that need more than HTML5 has.  (Which is not much.)  Maybe we can at least not have whole sites done in Flash.  The downside of this is likely to look something like the days of <blink> and <marquee>.  People (most of whom are management, not designers or developers) will be asking for the "whizzbang" of HTML5.  Please, resist the urge to implement things "because you can."

Opcode Caches

Let's be honest: Drupal 7 is not lightweight.  I've just developed a site for a client that is hosted on Dreamhost, and we're running into memory troubles.  (In particular, the new UI for Tokens is a killer.)  While opcode caches like APC and XCache don't fix that, they do eliminate or reduce the need to parse and compile the source for every php file on your site, be it Drupal core or contrib.  (Note that eval'ed code cannot be cached.)

Horizontal Scaling

More and more, Drupal is being used for high-traffic websites.  While Boost, APC, and core caching help a lot, if your site has a lot of logged-in traffic, they can only do so much.  You can scale vertically, but there's a limit to how big a server you can buy, and vertical scaling doesn't give you any redundancy.  While horizontally scaling your web servers is (comparatively) easy -- buy a few web servers and point a load balancer at them -- the database servers are harder to scale.  Drupal 7 has added some ability to have master/slave servers to distribute read load, but I hope we'll see more work towards support for full clusters of DB servers where you can even distribute the writes.

More Distributions

Just like Linux, Drupal comes in distributions.  There's the core Drupal distribution, but there's also OpenPublish, OpenPublic, COD (Conference-Organizing Distribution), and OpenScholar, among others.  Check them all out at Drupal Distro Watch.


If you're not already using Drush, you're doing it wrong.  Drush makes things like enabling/disabling modules, updating modules, executing cron, getting to a SQL shell, etc. so much easier than going through the web interface.  Plus drush itself is scriptable and extensible.  Oh, and you can do aliases for whole groups of sites (awesome for security patches).


What are your predictions for the up and coming Drupal (and web development) trends?

I have the coolest wife...

I have the coolest wife because, although today is our first anniversary, she not only allowed, but encouraged, me to attend DrupalCamp Atlanta.  Hopefully I learn something useful.  :)

I love her very much... and I'm not just saying that because she reads this blog.

(Oh, and don't worry, we'll be spending the evening together.)