Bigger Projects

Our first official programming project ever–the first practicum assignment in the first term– had a total of under 50 lines of (Ruby) code; in fact, the core logic was 11 lines, the rest was console output and testing. And no, that wasn’t “hello world”; it was a loaded dice simulation. Ruby is that terse.

After that, size and complexity of our projects increased steadily. In the second term, the notorious (and for me, at times, murderous) media player topped at roughly 4,000 lines of Java code. Now in the third term I am thinking nothing of writing 1,200 lines–all alone and within a couple of days–for a sub-task of a homework assignment in a math course. But very soon this will all look like child’s play. A week ago we started to plan our project for software engineering, part II, in the summer term.

To be sure, software engineering (part I) was always about the design of bigger projects–a lot bigger than anything we had done or seen so far. A total of 11 lectures about user stories, Kanban boards, requirements analysis, use cases, software architecture, design patterns, and so on, have acquainted us with the tools, but naturally primarily in a theoretical way. The practical exercises (and we had a lot of them) were willy-nilly rather limited in scope, so that it all looked very much like an overkill. Particularly once we got from formulating requirements (planning what a software should do) to implementation (actually writing code). As I related earlier, in the last two practicum assignments we had to set up a complete scaffolding for a database-backed web application, using the Spring (Java) framework for the backend, and Angular2 (a Typescript framework) for the frontend. Both communicated via a REST service, which basically means–for those not familiar with web technology or JavaScript–that the backend offers methods for getting data packaged in JSON objects (basically, a container of key-value pairs as text), and the frontend requests this data via HTTP calls and displays it on a web page. And of course vice-versa. Sounds scary? It is. An insane amount of overhead for an example project that did nothing more than simulate a ridiculously basic bank account administration. All you could do was submit a transfer and return a list of transactions and the current balance.

And to get even that far took me about 40 hours over the course of two weeks. Just getting the backend to run felt like feeling my way through a three-dimensional maze blindfolded. Spring is really built for designing huge projects. And our professor required us to use (or should I say mock?) the design for such a project: Separate the interface from the implementation, the persistence (database) from the entities (data objects), the entities from the use cases (behavior), and so on, resulting in a huge array of classes, even packages, for an extremely limited functionality. Just understanding how this was all supposed to play together, let alone getting it to actually do so, felt entirely hopeless for a few days. Most of the time I liberally sprinkled annotations all over the place (the parts of the Spring framework interact almost entirely by annotations, i.e. @-prefixed comment lines in the code), and hoped against hope that somehow the pieces would all fall in place. Which, after many days of staring transfixed at console output gibberish with incomprehensible error messages here and there, and spending hours on Stackoverflow trying to find someone else with the same problem, miraculously they finally did.

But surviving Spring was nothing compared to the agonies of Angular. Spring, afterall, is Java, and Java means I get error messages (albeit often buried among hundreds of lines of unrelated output), a working debugger, and can print statements to the console to try and see what my code is doing. Angular is Typescript, and while this is a subset of Javascript, and JS is not entirely incomprehensible, the framework itself is still an entirely new technology for me. Normally I would have preferred to get a book and start learning it from scratch, but there was no time for that, so I had to get some results by trial and error, fast. And debugging a web application does not lend itself to that easily. Basically, whenever there was the slightest error in either the Typescript or the HTML files, Angular would do exactly nothing. A Java application with a bug will still execute the code until the first ocurrence of the bug, so you can see what works and what doesn’t. But an empty browser window tells you not the least bit. I finally discovered that Firefox has something called a browser console that would contain some error traces, but they were nothing if not cryptic.

Even so, over the course of three or four desperate days, and by copying together code fragments from all over the internet, and adding some boilerplate from an example project provided by our professor, I finally managed to extract that ridiculously small amount of functionality mentioned above from the Angular mystery. The best I can say about this experience is that it pushed the limits of my frustration tolerance outwards.

In a way, of course, this was “hello world” on the framework level, and thus the first step on the road towards designing (and even implementing) much bigger projects–projects for which this huge overhead might actually be a help rather than an obstacle. In the last session of software engineering, part I, in mid-January, we are now requested to present our plans for the practicum project for part II in the summer term, and this project should, by default, be a web-based application using the discussed setup. Unless we have a good reason to use other technologies, the professor said, we should stick with Spring and Angular2. And until January we are to work out not just an idea for a project, but do almost the entire project planning as well, including user stories, a Kanban board, a first iteration and an estimate for the effort needed to implement it (yes, we are doing Agile development–we are even encouraged to pair-program), settle on tools and technologies, assign roles in the team, and so on.

That’s a tall order for a few weeks that include the christmas break and the bulk of the exam preparation period. And it sort of spoilt my admittedly vague plans to try and find myself a different group for the summer term. As I said before, cooperation with my programming partner always makes me feel somewhat patronized and harried. That’s certainly nobody’s fault: We are just an unlucky match of personalities. He is overenthusiastic in all things programming, and I cannot match his easy grasp of new concepts or the time and effort he invests in mastering them, yet at the same time I find it hard to just accept that he should get his way in every little thing. So we clash, but as a rule I get bested in the argument and then feel bad.  Also, since he is much more productive than I, very often programming together means I watch him do most of the work and struggle to defend at least a small niche where I can do–and thus learn–some things myself. In fact, that I fought my lonely fight with Spring and Angular was primarily so that I would have to do it myself, rather than watch him do it.

In any case, I often thought I might have an easier time with somebody more on my level, or at least more relaxed. Particularly faced with the prospect of a major programming project in which my programming partner would inevitably take a very active interest and hence would insist on his ideas and solutions. It’s bad enough with math assignments and such where he basically doesn’t care. For a few days I even asked people whether they already had a complete group for the summer term. Unfortunately they all did. At least those groups where I had the feeling they might be able to get their act together. For that, of course, is the major advantage of any group my programming partner and I are both in: Things will get done. I shudder when I think I might be in one of those groups where people start preparing the homework assignments in the last 24 hours before they’re due. Some people even present code in the practicum of which they know–and say–it doesn’t work. Personally I’d rather do something really disgusting (get out your Blackadder movies and find the quotation).

It might have turned out different had we been able to wait until the start of next term before we formed groups and planned projects. For instance, some people might have failed in the exams, or decided to leave the study program, or not do software engineering in the summer term, thus causing a reshuffling of teams. But the early presentation date basically forced us to remain in the present groups. So I was left with two other options: Either invest no effort at all in finding a decent project for our group, so we would do something really boring my programming partner would gladly let us do with the minimum effort. Or conversely, come up with a really exciting project that would compensate for the frictions we would inevitably have.

Turns out it was the latter, more or less incidentally. Our software engineering professor ended the last lecture before christmas nearly two hours early and encouraged us to use the time to discuss our project ideas. There were just three of us at that point (one of our group attends the lectures only very rarely), and our extremely luckluster brainstorming failed to turn up anything that didn’t make us yawn. The usual suspects for web-based applications are either social media sites or webshops or any mix of these, and that’s about as exciting as inventing the wheel for the umpteenth time. So I thought I might as least give an accidental idea of mine a try.

I have always been interested in transportation networks, and suggested we might do something in that line, be it a serious simulation or a game. Funny enough my programming partner was immediately interested, because he imagined we might use a graph library for the implementation of the network, graph theory being the one subject this term almost everyone considered both fun and useful because the applicability to a whole host of everyday problems was so obvious. So within minutes we were agreed to do this–or something like this–as our summer term project. Fortunately, the absent fourth in our group not only didn’t object, but was excited as well, because he is more than usually interested in gaming. And this project seemed like a good way to do something gaming-related without the nightmare of 3D graphics, realtime and so on. Of course, we could neither hope to compete with the well-established commercial city or traffic simulation games out there, nor offer an actual scientific simulation of urban transport problems. So we settled for something middle of the road, calling it a learning game–a gamified and somewhat abstracted approach to understanding the problems of moving people about in a metropolitan area.

I went home sort of elated because I thought this could be good, and over the next few days I contributed my share towards writing about 40 user stories that, if all implemented, would keep us busy for not just a term but probably a couple of years. Software engineering practicum the following week brought me down to earth again. After the assistant had barely glanced at our (my) hard-earned solution to the Spring/Angular/accounting assignment and given us the nod that signified we would get the Prüfungsvorleistung, we had three hours to discuss our project further.

It turned out we had very different ideas of what our software should do, and how. For instance, it was entirely unclear whether the user would be given an actual 2D map to plan his network on, or whether it would be completely abstract, with just a graph representing population centers and traffic relations. And a host of other questions followed from this, such as how clusters of inhabitants would be represented in the game–by area, or by number, and so on. Unfortunately we failed signally to actually discuss these questions. Rather, a clash of egos developed in which we were close to trading harsh words and I was just short of giving up on this whole project. My programming partner was impatient and tried to stifle the debate by riding roughshod over everyone who wanted to actually discuss the merits of alternative approaches; I felt cornered and unable to argue, as I am wont to; number three in our group was merely shaking his head; and number four tried in vain to make himself heard. It was a rather inauspicuous beginning for our project.

I slept badly the night after this. In a way this is how I had always feared a major project with this group would turn out. My programming partner would have very precise ideas and inevitably dominate the whole group, I would lack convincing arguments to suggest other solutions, even when I felt discussing alternatives might be worthwhile, and nobody else would much care. And we would have endless unpleasant debates right down to the implementation details, souring the whole project at least for me. Or sit, the four of us, in front of a computer, with my programming partner writing imcomprehensible masses of code insanely fast while we others watched in awe. I have seen that actually happen.

Now there were was one lever I had to try and prevent this. The setup for a major project specified by our software engineering professor included the stipulation that we should have formal roles–a project leader, a lead architect, a product owner, and a quality assurance manager. We had earlier had a brief discussion over whether we should assign these roles according to our own talents and preferences, or rather the other way round, as a didactic exercise, i.e. let everyone try something new. In order to prevent unneccessary friction, we had agreed on doing the former. That meant my programming partner became, of course, the lead architect, our gamer the product owner (proxy for the–imagined–users), and I the project leader–I have traditionally been the organizer in any group I was in.

In that capacity I now suggested rules for a division of responsibilities that would make sure that we kept out of one another’s hair as much as possible. The project would be divided into losely coupled components–in keeping with what we had learned in software engineering!–, and each one of us would be primarily responsible for a few of those. Interfaces between components would be discussed only by those involved. Debate in the whole group would be limited to the major design decisions. Of course, actually implementing user stories (representing features) would still cut across this separation of concerns, necessitating some coordination, but in doubt the people responsible for the components would have the final say. I thought this a great way to make sure this project could be implemented without having to discuss each and every line of code–particularly with my programming partner–and with a chance for all of us to learn something new. Not to mention that it actually made sense.

I was still sort of surprised that these rules passed without any great debate this morning. In fact, my programming partner was the first to agree. I had been prepared for a long debate, even for making acceptance of these rules the precondition for my further participation in the project. But of course I am glad that having a reasonable division not just of labor, but of responsibilities as well apparently made sense to the entire group. No doubt there will still be a lot of discussion. The core components of this rather complex simulation are so closely linked that they will still involve all of us. But hopefully only to the point where we have reached a consensus. And then we can go and code more or less independently. This way, the project might actually be fun.

And of course, in the end it will also be a good thing to be in a group that gets its act together. To the best of my knowledge, no other group is even close to having more than a rough idea of a potential project. We, on the other hand, could do our presentation tomorrow, if necessary. We have our roles assigned, both global and components, we have a wiki with a several-page description of our project down to architecture, technologies, and simulation model, we have a large number of user stories, and a reasonable idea what our major  components should do. We could start coding after a few more hours of debate. In a pinch, without.

And mind you, though being project leader makes me responsible for actually getting this show on the road, and that will, again, mean most of the coordination will be between my programming partner as the lead architect and myself, I do feel that in doing so I am–we are–learning something that might be relevant to eventual actual projects in the real world. This project is likely to get a lot bigger than anything we have ever done before. Maybe big enough that the entire setup, from framework technologies to a component-based architecture to a project structure with fixed roles, might not actually be that much of an overkill any more. In a way, this is just starting to feel like we might soon be playing with the big boys. Which of course we eventually will. Actually it is kind of exciting.


Leave a Reply

Fill in your details below or click an icon to log in: Logo

You are commenting using your account. Log Out /  Change )

Google photo

You are commenting using your Google account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s