L.gorithms

7 Technical Management Techniques

2011-06-21T20:50:00.000-07:00

Often it is the case, where team leaders and managers at software companies are ex-geeks that climbed the ladder to reach a managerial position. Well, "climbed the ladder" is really a misleading metaphor. First, who says managerial positions are higher, or better, than development positions? Some would argue the opposite. And indeed, in some companies, managers are offered optional rotations of management and hands-on R&D. Second, it is misleading since the less glamorous reality is that when companies expand, someone has to manage the newcomers, and those new managers where there at the right - or wrong - time.

Either way, geeks being geeks, social codes and human interaction were never their forte. How can they manage other people if they can't get a date without a facilitating website?

So, let me tell you some secrets. First off, the words 'Technical Management' in the title isnot what you think. You might think it's about management of technical people, but it isn't. It's really about the techniques of management. It's about knowing and using several technical and emotional rules, that would make the life of the manager geek (you?) so much easier, and would get results with less friction.

However, these rules and tips would not do the work for you. We are talking about Technical Leadership, but this post is more about the Leader part, and less about the Technical. The technical side deserves its own article, or maybe a book.

But there are some intersections. In your technical leadership, you can decide to follow either the Surgeon model or the Decider model. A Surgeon leader works as if he's a doctor operating a patient. He has a full staff - from nurses to residents - helping him do all the work by himself. The surgeon cuts, analyzes, takes a direction and on most cases, also stitches.

On the other side, a Decider (if I may quote someone) doesn't really have a concept or an technical ideology. It hears the opinions of the team, asks a lot of questions, and decides, this way or the other.

These are two well-defined extremes. As for you, you probably position yourself somewhere on this range - for better or worse.

When it comes to leadership, one needs to remember that management, as any other profession that deals with people, is highly complex and no recipes are to be found. One needs to understand the platform, the substrate for everything. And this platform is respect. This is what everyone is after. I think it's so important so I'll repeat it: respect is what everyone is after. Respect (self-respect included) cannot come without its sibling: humility. So if you have respect and humility, and you should, all other things will come easily. You will exert politeness, consideration and gratitude. You could give remarks and criticism without guilt and without being afraid of de-motivation. Most importantly, you will receive respect in return.

Now, after this short introduction, let's cut to the technicalities, to those techniques that would help you transfer this sought-after respect:

Technique 1: The Oreo

As Noel Coward once said, I love criticism just so long as it's unqualified praise. No one likes to be criticized. on the other hand, critique can be a gift someone receives as an opportunity for improvement. And you will need to criticize your guys since you are probably would have, sometimes, different ideas than them. So make them an Oreo sandwich: start with praises, put in the critique, and end with some more praises. Something like: the banner design is really cool. I think though the middle block should use Ajax, and not PHP as it is now. Still, overall the new node looks awesome!

Technique 2: No Buts!

But is a traitorous word. It tends to minimize whatever came before it, and sometimes simply nullify it. For example, when you say something like "listen, this piece of code is elegant, but the performance is bad". What the poor guy is left with is that the performance is bad. And thats taking a hit after investing so much thought in making it elegant (hey, elegance in code deserves another post altogether). Yes, she would survive, but why chip away in the motivation, if you could just use another word instead of the 'but'? What if you would just replace the 'bug's with an, say, and? Imagine now how the sentence would sound like, and tell me this is not a brilliant technique.

Technique 3: do you know the real meaning of assume?

Many a time we tend to think the other person is just an extension of our own self. This belief can be found in couplehood, as well as at work. And at those two places its exactly one of those beliefs that complicates matters. So here's the latest: The other person has different opinions, different priorities, different ideas. So when you assumed that she was going to do something in a certain way, your mistake was that you didn't convey this assumption to her and transformed the assumption into a discussion. Because when you assume, you make an Ass of U and Me. So Talk. Ask. Discuss. Just don't sit there at your desk and assume things will go right.

Technique 4: Praise in public (condemn in private)

With today's open space environment, the use for private areas is even more in demand. But dont make the mistake of using it for telling your man how much his work was good. Try to find any opportunity to praise him in public. And the opposite is true as well. Don't let him look like a fool. It's bad for him (or her), and it's bad for everyone else too: do you think they really believe its saved only for this poor guy? They are very much afraid for the moment it will get to them!

Technique 5: Responsibility goes with authority

Remember how they told us to delegate? well here's a quiz: how can you delegate and still be the manager? The answer, my friends, is that you can't. When you give someone the responsibility on a certain subject, he's the manager. And this responsibility must be accompanied by authority, too. The authority to make decisions, and the authority to make mistakes. Because those who don't make mistakes, are those who don't make anything. If you wan't to delegate, you should want that they will make some things.

Technique 6: Save the perfectionism for yourself

If I may choose only one technique, that would be it. Oh, how many managers did I see that thought their demand for perfectionism serves the cause. Here's a news flash: you aren't perfect. And your job, as a manager, is to understand that no one is perfect - but not to accept it. Remember: everything we have today, is a result of gradual improvements. So when the next engineer declares a done task, test it, compliment him, find the flaws, and continue to the next layer of improvements.

Technique 7 : Blame has three fingers pointed at you

This picture should be remember by everyone who feels the urge to blame:

As everyone can see, when you point a blaming finger and accuse someone for doing a stupid, stupid mistake, what really happens is that one finger does point towards the accused, one finger points up to bad luck, and three fingers pointing at you. Get that? you do the math.

Be humbler!

2010-08-13T08:06:00.000-07:00

I think it's pretty amusing to see how many applications jump when you try to exit them and immediately ask you 'Are you sure?'. If you say 'yes' some would add 'Positive?' '100%?'

It's a clear case of developers' separation anxiety mixed with hubris. Guys:
a. Be humbler
b. Get a life!
If the user quits the application, it's most likely not by mistake. She really wanted to leave. So, you're not really helping the user by posing yet another message box, another movement of the cursor, asking for another click. Let her go and God Willing, she'll be back soon.

The charm of the bourgeoisie or: are we going backwards?

2010-03-05T18:36:00.000-08:00

I'm not sure how wise to let the whole world know, but my favorite music recently has been Arabic and Italian. What a mix! Kudos to Last.fm. Their music collection and the radio stations they offer is just superb. Nagat El Shaghira is so cool!

So, it wasn't so surprising that when I saw Last.fm app offering on my XBox I clicked, installed, and immediately had all my beloved stations - Ya Habibi - on my Xbox. And from this simple experience, I have gathered several interesting insights.

First, that game consoles are becoming like the iPad (hey, it's Pad, not Fad) devices, and vice versa. The process of getting a Last.fm station was strikingly familiar to iPhone app installations. Select -> Install -> Run -> That's it. Hmm, and where is the keyboard? yes, the lack of a real one is also a surprising similarity. Oh, and what about the one-app-at-a-time concept? Wait a minute! there's something here. What's going on?

What has been going on is the second interesting fact. The Back To Basics movement has been with us for some time now. Operating systems, applications, and more than all, programming teams around the industry have started to focus again on Computer Science 101 - improving performance - and, God Forbid, on dropping features. Here, we see even a stronger tread backward: back to single-tasking operating system, back to primitive input devices, back to focus.

But this is only natural. And natural is the keyword here. Some of the user-facing people understood, finally, that computers has become too complicated. If not for us, then for our mothers. And it is our mothers, our girlfriends, and our kids, that they want to get a new market slice from. And for them, focus is one. They know, that they cannot drink and drive: they cannot multi-task. They can only do one thing at a time. They don't need 102 keys on the keyboard, if they only want to listen to some cool music. Just like when they flocked to google.com, it relaxes them - and me too - to see a single user interface control on screen. Simpler is super.

All of this is just another chapter in the inevitable journey of the PC from the home office into the living room. You have to make it friendly. Like the charm of the bourgeoisie, it is simply -- convenient. Simply friendly. As Nagat sings: Shabna, Shababna. Our friends, our youth. Friends must come easy -- and simple.

Forget performance. Let's add another feature!

2009-07-27T20:37:00.000-07:00

Sometimes i think I'm a Benjamin Button kind of guy. As I grow older, I tend to do young, stupid things. For example, I have carelessly dumped my old Firefox in favor of a new Chrome browser. Then, I threw away my Palm Centro for an iPhone and reformatted my hard drive after my laptop slugged away for weeks. And as I done all this, I found out there's no turning back; and sometimes you feel you've lived your whole life in ignorance.

Google has marketed its browser as the fastest browser around today. Even if the performance is not as good as they claims, the minimalistic feeling certainly makes you feel this way. Apple didn't have to market anything. The amazing 3D effects in the iPhone's user interface and the smooth application performance market themselves. With current Vista performance I tried linux a few times although I quit it when gopc.net didn't allow me to install any applications.

All of these performance upgrades have one thing is common: out: new features; in: a lot of performance boosts. And here's the big question about all these performance upgrades. Where have you guys been all of those years? Have you forgot about us? Have you been deserted in the feature-land with all bridges burnt? Sometimes it makes me wonder. For years, Microsoft have added more and more and then more menu items on its Office applications - just enough to confuse you and obscure the real important ones. And who remembers the 'Offline pages' feature on early IEs? or the newsreader that came along with them?

I can't blame all those product managers who succumbed to the feature pressure. The "just throw more hardware at it" approach used to work on some situations, and was definitely enough sometimes. And Moore's law predicted that this trend will continue, maybe ad-infinitum. One might think, that it didn't reach the good developers. Well, you are right about the good ones. But all the rest - and more so, their managers - completely forgot to ask the right questions and never got the time to architect it right. Let's get it out there, fast, they said, and then let's get the next version, too.

But not any more. In a competitive environment, finally, finally, some companies are making products that at last gives me the feeling that I'm sitting next to a modern computer.
As we saw how Windows Vista is being actively cut back (can you imagine, features actually dropped); as we saw browsers like Chrome take off with an easy start; as the iPhone exploded with popularity, we knew: performance is back. And I hope it is here to stay.

The Dilemmas of Coding and the Architecture

2009-01-10T21:55:00.000-08:00

I found a great blog about software architecture, and in it, a great post. What I like about this blog is the emphasis they put on the inherent conflict of the architect. The daily struggle with the developers who want to code now, and think later. The conflict with the managers - and the budget - who need to see the project delivered already. But more than all, it is about the fight with yourself. There are many such conflicts, some subtle, but definitely they keep the stress up and the spirits low. But in the end of the day, I sometimes feel, it's about the internal dilemma that involves the need to both manage your project - all of it, leaving no corners unwatched - and in parallel, the need to code the core architecture in the way you see it.

In Hebrew there's a saying: Tafasta Merube Lo Tafasta. Which means: if you try to catch everything, you will catch nothing. I learned time and again that you can't catch and control everything, and if you try to do so, it will have the reverse effect. On the contrary - you must let go, must delegate responsibility and give with it the authority to do so (because responsibility cannot go without authority). This, my friends (as McCain used to say) takes courage and calmness.

On the other hand, an architect must always code something in his project. Doing so will you can give you the feel for the code, and more importantly, the feel for your programmers. You will understand the daily hurdles they have to tackle, and it will free you from being the frustrated perfectionist, sitting in a managerial ivory tower surrounded by Gantt charts. That's why I like the name of the blog so much: Coding the Architecture.

5 frameworks every software architect should consider

2009-01-05T21:09:00.000-08:00

One of the best things I like about developing software, is that every project is never like the previous one you've built.

That's right. But underneath all these systems, lies a set of common architectural modules, that every system should have. If you are an architect with a team of developers, or one developer with a team of managers, here's a list of a set of modules, frameworks and strategies you should have in mind. Not all of them would necessarily apply to you, but if I were you, I would at least think about each one. So if you are starting a new project - lucky you - from scratch, or coming to an old project with a mandate for change, here's a small checklist.

ORM Framework
Many programmers would start their efforts with writing a Data Access Layer. But nowadays, it is becoming increasingly clear that not only it is important to have a DAL, but a layer with objects (that is, classes) that maps to database entities. This is ORM.

A good ORM layer would have 3 minimal features:
1. It should allow automatic creation of objects. This is a nice example of the DRY principle. All the database-related fields are defined in one place: the database. With one click of a button - running a script - the classes are automatically created. This is code you don't touch. When you change one field in one table, you run this script again, recreating the whole body of classes once again.
2. It should support record CRUD - creation of a new record , updating of an existing record, and deleting of a record - all as operations availabe on the classes created.
3. It should know to deal with relations between tables. For example, if you insert a new order, you usually need to insert the order header table, with the master order details, and then a list of recorders int the child order details table. Traditionally, you would insert a new record in the master table, and then take the key that was created and use it as a foreign key for the child records. A good ORM would abstract that for you. For example, in Propel you would write:
...
$order->addOrderdetails($orderdetails);
$order->save();

and this last save() will insert the whole tree - even more than just one level deep, by the way

Tiered Framework
Once there was the 3-tier application, divided into the user interface (UI) layer, business logic layer, and database layer. The ORM is obvoiusly the database layer. Now the question is, what on earth is business logic? I always find this term a bit vague. When you build an application, what is not business? I think that the answer is, if it's UI, then it's not buisness logic. To add to that, business logic usually includes back-office processes such as automatic emails, pricing calculations, conversions, and more than all - decisions. For example, what form to present next.

What would such a separation give you? In web sites, this tiering would allow you to change the UI easily without changing the underlying, um, business logic. If you are talking about a desktop C++ application, then the benefits could include - how exciting - a cross platform for Mac. wow!

How to do it? in general I would leave it to you and your framework. I could say, though, that in my web sites the mechanisms I've created simply put all the UI related code in certain folders and classes, from which the forms and html have been built. In other applications I've used a tree-like module structure, where modules are interconnected by subscribing to events (more on that later) that each module can publish. The UI forms are simply modules in this tree.

There is a third way of separating the UI from the business logic and the db. Remember that you UI is a tree of control each containing smaller controls. For example, a form that contain a grid whose some of its cells contains a calendar control. Theoritaclly speaking, you could have a real tiered application who would have 3 tiers, but they would be separated on every level of the UI tree. In that scenario, you would have a business object for the form, that will contain a collection of objects for the grid lines, that will contain a collection of objects for the grid cells. Each of these object would have its related UI "tier" - meaning, a control that is bind to it. It would also have its bound database entity, but with tabular databases, the question is, how to represent the relations between the objects. And I have no clear answer for that. Maybe I'll try it on my next system.

Event-based Framework
I know this might be seen as an overkill to some, but hear me out. why events? This is simple. An eCommerce order form (AKA 'shopping cart'), for example, can be viewed as one module which includes the order date and number, the customer and a collection of the products on this order. The inventory, on another side of the system, is another major module holding all the products and their quantity in the warehouse. When an order is placed, you would want to update the inventory with the products sold, so the quantity in stock could be decremented. The simplistic way would be to have the inventory module exposing methods, and call them directly when the order is placed. The problem with that, is that some day - and you can count on this - some one will come up with a new requirement for more modules to be updated on an order. For example, an email module for sending an email to the manager reporting the sale. Now, for every new module that needs update, the developers will need to change the order module and add a new call to the new module. While the young, talented programmer looks at this as a non issue , we experienced architects know how much it can be simplified with an event-based framework. In such a framework, placing of an order would publish an event to the whole system. Not only would the Inventory module subscribe to it, but any new module that wishes to act upon this trigger, would subscribe too. So the amount of work needed to accommodate new requirements is minimized at least by half: there is no need to touch the order module. In reality, since software complexity tends to be exponential, the decrease in work is in much, much more.

When you do go ahead and write your publisher-subscriber framework, remember that you might take one more small step that will pay off big time later. As I explained in the "Tiered Framework" section, every system is inherently based on a tree-like architecture. Imagine, for example, that a sudden price change occured for one of your products. Different requirements arrise, one of them being the need to update all orders in queue that contains this product. A simplistic approach would be to search all these orders, find the ones needed a change, and update the prices.
But then, when a new requirement would arise, again we need to touch the source.

In a tree-based architecture, the orders are children of the root, and the products (also called 'order details) would be children of the orders.

An event would be published to the root. The products subscribe to 'price changed' event, and if an action needed, call the parent order (or better more, publish an event to the parent order) to change the total, the presentation, and do whatever else is needed on such event.

By the way, the same logic works in much more interesting applications - games for example. If a missile was fired in the area, all subscribed NPCs (Non Player Characters) would want to know about this event, so they could evade - and die, if hit was decided. Other modules would like to know that the missile inventory should be decreased, that the event should be reflected in a remote multiplayer session - in short, there is no shortage of wars to win with this frame of mind.

Cache Framework
Maybe one of the most important performance optimizers. From the L2 CPU Cache to the database's memcached, everyone should have a cache. To be honest, I'm still puzzled as to why it really works so well, and still at awe watching the magnitude of improvement. But it is clear now that this is the biggest performance boosters ever created. And every system should have its own Cache Manager class, which centralizes few important actions: from adding a value to the cache, to retrieving, invalidation, and - very important - single point of overall cache deletion

Error and trace Framework
It is sometimes amazing how this simple module often get ignored until the later stages of the project. But a clear definition at the beginning of the project can do wonders to your ability to find fast bugs and exceptions.

There are some nice open source pacakages that offer a nice set of features. But whatever way you implement it, here are a few key notes I believe worth repeating:
1. distinguish between tracing and error reporting. It is not the same. Tracing is used for the development team, or for very problematic client sites. Errors are something else - you need to always report them
2. an error is not a bug. If a function cannot calculate a price for this quantity, it might be reported to the user, but not an error. In my view, every error is an Exception. Something that no one foresaw.
3. Invest the time to get real time tracing. You'll thank me later.

Every software system is the unbelievable sum of its developers. These are some of the frameworks that will hopefully help them succeed, and their system to grow without collapsing under its own weight.

Views on Misearable Programmers and ORM

2008-10-03T19:51:00.000-07:00

I think that there are miserable programmers. Those who need to deal with database issues, for one. At first it feels good - every new table connection is a added ornament to the party. Then comes another connected table. and another one. soon enough, it goes out of hand and everyone, from the project lead to the last programmer, is miserable.

So that's why we have the ORM. I love it. For each table, you get a class. So I've introduced it to one of the girls, and thought, that would not give her a chance to act miserably. Oh boy, was I wrong. It turned out, that the ORM just made this too easy. Because the connection to procedural programming language is so direct, she accessed the database as if it was an array in memory. one line gets the first table. Another line iterates on the records, and for each one gets the child record. That's how you can get a perfect siesta, because you can get some sleep while the loops finish running. So lame.

On the other hand, the ORM we were using - Propel for PHP - was not supporting easily a custom join SELECT (and I doubt if any ORM would). So the idea that saved the day was to use a VIEW. With views, you get the good of both worlds: the ORM treat them as tables, so you get a class per table. and the performance would be optimal, since all the joins are done on the SQL level. And the best: you dont need to write a stored procedure.

This reduced the code level a little - from 20 lines, to 5. The performance is obviously better. and now she's much less miserable, and so am I.

Programming as a science

2008-04-02T19:57:00.000-07:00

I heard there something called Computer Science. I even have studied it at one point (4 years, to be exact). But now I think, that it's not really a science. The real Computer Science is just beginning to form.

A science is suppose to explore nature. To understand and define governing laws, and to build formulas to predict effects from causes. But the CS I studied was all about inventing clever -- too clever sometimes - algorithms, building useful data structures, but it was not about exploring nature.

I'm the founder of GameX, a system that allows you to record movies out of games. We do it by injecting code that re-routes DirectX calls into our DLL. I've read some articles about such subjects, saw an interviews with Mark Rusinovitch, and gradually I realized that only now we are starting to approach scientific adulthood.

It is because now, with the maturity, stability and widespread usage of Windows, or Unix, these systems have become part of nature. Programmers today really explore them - their behaviour, their rules and their exceptions. They taught you on the university about algorithms and data structures, but really, working in the industry is so not about that, and so much more about learning to work with the quircks and rules of every domain you found your project at.

Very seldom does a programmer today program just in C++ or C#. Much more often, he or she has to interface with a plethora of libraries, and explore their behavior. You cannot change the way ODBC works, you can only learn how to deal with it. Yes, theoretically you could have written it yourself, but realistically, no one has the budget or the time. And it applies to ODBC, DirectX, DirectSHow, FFMpeg, Wikis - you name it. Whenever you turn, the software libraries reign and SDKs rule.

Now when you are dealing with a hacking project like GameX, it is even more significant. You cannot change the way Windows works. You can only learn it. And much like the scientist cannot see beyond molecular level, you don't have direct observation methods either - which in our world, would translate to a decent debugger.

But the real question still lies ahead: Is it a bad sign when a field turn into exploration rather than invention?

The Power of Programming

2008-01-20T20:55:00.000-08:00

Deep down, I'm still the same programmer that I was when I was sixteen years old. Recently I've started to realize how boring it can be to be a manager. I was a manager way too long. It feels powerful at first, but after a while you realize all you really have to do is keep track of the project plan. It's basically like the inventory work, like of a warehouse guy, just instead of boxes you have features. If you're good, you remember all the boxes, where they are, where they ought to be, and you're sole job is to keep them organized. The other thing you have to organize is the people. They call them 'resources' in manager lingo, not people. And we're not even ashamed of that anymore! I guess when you just start to be a manager you're kind of concerned that someone would be offended if he'd know your'e calling him a 'resource' but after a while it gets to you. So you organize the features, and the resources, and if something goes bad, you can replace one feature with another or one resource with another or if you're really creative you can exchange some resource for a feature. How brilliant.

but coding, man, something else. Yes, you have tricks, definetely you do after a while, but the real nice thing is that actually sometimes tricks won't save you. You'll just have to be creative and invent some class or data structure that would do the trick in this very own particular place. And if you don't know what I'm talking about, I guess you won't be visiting my blog too often in the future.

Am I singing here for coding? for sure. because this profession deserves more fame. Should be all out in the arts. I'll wait for that, while show biz continues to make movies about show biz and some other brilliant professions.

Open Shmource?

2007-08-07T22:51:00.000-07:00

A new project has come my way. Extremely interesting, it involves time-critical synchronization between processes on different machines. Isn't that cool? Pushing the Western frontier out of the box and into the realm of distributed computing. Do-it-yourself Grid.

For the synchronization, we decided to create a communication module that would be used to send packets with time information across the network.

Sounds like a good plan, but recently, I've started to have a continuous internal debate between two development methodologies. The first, represented by my friend 'The plugin guy', says - buy a component and use it. It reduces your risk, since under a finite budget and time line, you get a ready made module to use. And not only would it be there for you now, it will keep improving, because the company who created it initially will probably continue to provide upgrades.

On the other hand, whoever worked at Bloomberg have seen the complete opposite: these guys are writing everything themselves, from the network router to the database to the application. Why? Because, so says the founder in his book, Bloomberg on Bloomberg , it far less risky when you do it yourself. You are in control. You lead, not follow, and if for nothing else, its more fun.

Which method is the right one? where should I lean?

As always, it depends, but I believe being in control is better. I usually find a middle ground here. I don't start from a blank file, but rather, I go to Source Forge and Code Guru. I see what samples people have put there. download a few, compile, examine. Them I take the best one, tear it into its parts and reconstruct it in my own classes. I know, it's not totally Open Source. I know, .NET and Open Source are not the most friendly team mates. But recently I started to call it Open Shmource, and that's the only way I can use other people's code, whether packaged and compiled, or readable as an open, and very boring sometimes, book.

Living on the edges

2007-07-21T06:36:00.000-07:00

Many years later, I still remember how Professor Bar Yehuda taught us what he thought to be the right way to program. First identify the main problem, he said, put it in the center, and then start to build around it. Let's say you need a program that would present the multiplication board. How would you go about it? Your central problem is obviously the multiplication:

product = x * y

and then - only then - you should start dealing with the board

for x = 1 to 10

for y = 1 to 10

product = x * y

But now I think that sometimes, such an approach can lead to monolithic architecture and spaghetti code. Lately, I was involved in designing and building an eCommerce system, and I keep asking myself whether we should have started our design not from the core modules, but from the boundaries of the system. Whether we should have started from looking at the edges, and only then build the center.

An eCommerce system is one of the oldest software systems in use. A relentless textbook example, it is all about taking orders and passing them through a process of fulfilling, billing, shipping and sometime post-shipping actions like returns. It always amazes me to see how, after fifty something years of the software industry, the world still needs new order processing systems. Even more amazing is how much this can still be fun and interesting.

When I first joined the project, it was already a year into development. Order entry forms have been already programmed, as well as other important entities, such as products and customers. It was done in a pretty simple way, because building an order entry system is really a no-brainer. Take the data from the form, pass it to some stored procedures, and write it to the database. Let's first show the executives some working features, calm down the organizational anxiety, and then proceed with the rest.

Aha. As everyone know, the rest is where the problems never rest. With every presentation of the system, we were hit by additional requirements. Audit everything, they said - every action the user takes has to be logged. Build a rules engine, they suggested, that could, for example, check every incoming order and send an email if its amount is greater than $1000.0. Oh, and we need to export every transaction to other databases, like Quick Books and sister Web sites. When, asked the suits, do you think you will be able to deliver that to Sales? because we can't sell just a simple system. Everyone expects more from us.

Ok, so we had to start thinking about adding all that. You might say that it is still a no-brainer. Take the export requirement, for example. All you need to do is to build a system that takes the records from your tables and translates them to the external database format.
That's right, I would answer. But then, you would have a reference to the table fields in at least two places. The first place is in the form's stored procedures, the ones needed to load and save the data. The second place is in this new translation layer. Every field we add or change would be updated in these two places, a clear violation of the DRY principle. Young programmers might not think this is a big deal, but if you've seen some things you'd know that this is a recipe for a disaster.
Numerous other examples could be given. The rules engine, the audit module, and many more, can all be programmed using a simplistic approach, and they all would lead a working system that will smell, taste and rot like old Fettuccine because no one will be able to touch it once its cooked.

So should we know all the requirements from the get-go? Well, it depends. Under stress, some architects might resort to asking to get "the final list of all the requirements", just to fend off some of the pressure when this list won't be coming. But this attitude really wouldn't help us. In addition, there are anyways so many requirements to sort out, especially for new projects, that everyone's brains are working double shifts just to get their arms around the whole story. So thinking about anything that is not totally essential now is a waste of time.

But it really isn't. If I may suggest to everyone, myself included, to stop for a minute on such times and try to think about the boundary conditions. Not that it's easy. I found out that sometimes I can't think about it, because I get stuck. Then, it usually helps to talk about it with selected peers. To ask about this field and that field, even if they are pretty obvious, usually makes problems float to the surface and soon thereafter, some solutions might start appearing too.

Still, with all honesty, sometimes even that is not realistic. Not for nothing do we delay the less urgent problems for later -- it's might actually be a wise decision for survival. During a heated discussion about the rules engine, in an effort to embrace the new Microsoft Windows Workflows Foundation, I told one of my architects that he has to think ahead, about the day after the release. Lior, he said very seriously, if I don't get this rules engine in time, there won't be any 'day after' for me. And you know what? he was absolutely right. If we look too far into the horizon, we might not see the pitfalls that is under our feet, and we won't make it there.

So what should we do? Well, yes, it's true that you cannot anticipate what's coming and even if you do, you simply can't get your brain around it. But you definitely can, and should, design for a change. Design for a change doesn't mean adding spare fields to the database, or stub methods to your classes. Rather, it means that you should decouple the modules as much as possible, find a way in which modules could be added, removed, or replaced with as low impact as possible, and see how you can change your data types from a single place.

The right way to do that depends on the project. For our eCommerce system, we introduced a beautiful event-based system, where each module subscribes to events published by the principal forms. With the years, I discovered that event-based, publisher/subscriber architectures always win the day in allowing solid yet flexible base for a system. So I suggested it here too, and the team accepted. Because I joined a year after the project started, it was too late for some modules, and not all the developers embraced it fully. But still, it does lay the foundation for the whole system, on which I'm very proud. But this is a topic for another post, if not a whole article.

The Architect's Feelings

2007-07-14T19:34:00.000-07:00

One of the more interesting transitions a programmer undergoes is when she becomes a manager of other programmers. It might happen in different ways. Sometimes as a process with a healthy pace. Or overnight because the company needs a manager and you just happened to be there. But wherever observed, it is a fascinating path that involves strong, sometimes turbulent feelings.

Many aspects of this process have been explored in management books. How to build a team, how to cultivate the team spirit, how to create a community,respect your programmers, avoid micro-managing, be courageous, recognize without soapiness - all true and extremely important aspects, but all could pertain to any project management role in any industry. There is one dilemma, however, that is very unique to software architects. That is, how to translate your feelings into architectural explanations.

What do I mean translate feelings to architecture? We architects don't deal with feelings. Haven't we came to this field - or brought to it - because we stayed away from all those elusive concepts, multi-faceted, never well-defined, called feelings? Haven't we always tried to ignore, if not mocked all that nonsense that belongs more in the Faculty for Scandinavian Literature than here in CS, the Computer Science department?

Well, when I say 'feelings', I have to explain that this word means to me two things, somewhat related but still different. First, when used by psychologists, a 'feeling' would mean 'emotion'. It could be anger, happiness, enthusiasm, or any of those amorphous states of the mind, which a person posses, exists inside of, and which would usually paint the way he or she experiences events at that moment. Probably such feelings would be categorized as either being good or bad. Some religions, especially South-East-Asian kind of religions, try to teach their followers how to reach a state of no feelings at all - surprisingly a hard thing to do. Funny enough, this lack of feelings is always an experience that makes many people very, very - hmm, well, very happy.

But when I say 'feelings' I don't mean all that. I just mean, those things that you don't know how to explain in words. The elusiveness is still there, but its certainly not about emotions - its that accumulation of past experiences that paves the way you design your code and your system. It is the inexplicable driving force that dictates the way a programmer codes. And for the architect, it is a set of many reasons and rules, amalgamated in a strongly felt opinion.

For this architect, this poses a difficult dilemma. How to explain your opinion that a certain thing needs to be done in a certain way rather than the other?

There are several types of architects, and not every one of them would face this dilemma. To start with, there is the architect who does not posses any deep feelings about the code at all. He got to be an 'architect' because he wanted to manage, and the company needed managers. He doesn't care because he never cared - he really focused his career on becoming a manager, not a programmer. In fact, he might have been a programmer for a really, really short period of time. Since he manages a team of programmers, and he has a programmer's background, he is the architect too.
For him, what's important is that the code would work. He does not need to think about explanations. This module doesn't work, need to be fixed, please fix your bugs, and if you have some time left, could you perhaps take a look on the bug list and pick some of Michael's bugs, too? pretty simple.

Then there is the architect who is still a programmer. He cares, oh he really does. He might even care too much, and that is sometimes the problem. He cares so much, and enjoys coding so much, that he took on himself to code some of the modules himself. He might have just became an architect, still programmer at heart, and he does his game well, taking great pride of it. So when the project gets into a thunderstorm of bug attacks and delays he jumps in to save the day, solving these bugs himself. So he doesn't need to do any explanations either. He just codes.

But then there is the third kind of architect. This is the architect who is still a programmer at heart, but has no programming studio is at his fingertips. He might be an architect whose project got so big, that he needs to manage and couldn't afford to program any more. Or the architect at the start-up company, whose project is still small, but the resources are low, and he has to make up for the understaffing. He has to be the business analyst, the tasks distributor, the project plan owner, the tester and sometimes even the company technological evangelist -- so his time of programming is very limited at best.

For this kind of architect, a battle awaits. He has enough experience to know that a system needs to be built in a modular, loosely decoupled fashion, with just the right amount of componentization and a framework that would support - and promote - code re-use. His programmers know design patterns inside out; they were tested on it in the initial interviews. What they don't know, and he does, is that the system can't be just programmed. It has to be based on all of the above foundations, or else it will collapse under its own weight on doomsday.

But these foundations are pretty elusive too. They are not soup ingredients that we just add to the mix and lunch comes out great. Certain data structures and certain algorithms needs to be built, that would serve the needs of that very specific project. Usually, this architect would have used the old Nike methodology of project management: Just Do It. He would have coded the framework himself, and explain later. The code lines would break the language barrier, explaining themselves to the programmers. But this time he can't.

This time he has to explain.

And this is one of the most difficult challenges to many architects. Whether not coding at all, or whether coding a little and managing a lot, they have to find a way of explaining what they feel on how the code should be written.

Many beasts need to be subdued. First, there is the ego. Your ego and your programmer's ego. You know how much you hated when managers - usually of the first or second above types - tried to tell you what to do and how to write your code. So you don't want to create the same resentment here. Sadly is that sometimes you just avoid talking to some of the developers which you sense have the biggest ego, or the more fragile ones. Actually the latter guys might be your best programmers.

Second, there is the time problem. As an architect with not too much time to program, how could you have time to explain? Probably you too are running after the bug fixes, the clients new request, the deadlines - or maybe they run after you? In any event, it's really not easy to find the tranquility, the isolation, the quiet hours to sit with your programmer and to explain.

And third, not always you know what to explain. With the Just Do It methodology, you have programmed many modules by just doing them, starting with a concept and improving it over time. How could you do it with your programmers when you don't even code in this project? To make matters worse, because you don't code, you don't know enough about the real problems of the particular system and might offer ideas that are not realistic. If you recognize it, it might instill fear whether you could really affect the architecture. If you don't, your programmers will listen to you, say yes, boss, and do whatever the hell they want to do.

But in all cases, the problem is the explanation. How could you find the right words that would describe the right solution, be listened to, and make an impact?

We could try to give some answers to that. We need to know that not necessarily they would be encouraging. In fact, they might not be important. What does it all mean?

The first answer is that it really can't be done well. A good project should have clear separation between managers and architects. Between hands-on'ers to paper pushers, we might be tempted to say - but not really. Good projects should simply give their architects enough room, enough meditative time to code the frameworks, the architectures that would support the system and self-explain themselves to programmers in modules to come.

Second answer is that if needs be, there is a way. I saw architects who didn't program a single line in a project but still could explain how the events flow and what each table field means. These are the ones that demonstrate that you don't have to code in order to understand, and that an architect role is sometimes to be a junction of knowledge first, and then a source of it.

What these architects do, is ask. Curious as a child, they move from programmer to programmer and look on his or her code, trying to understand each set of mind. Especially if they arrived late in the project, but also if they've been there from the beginning, not always an architect could grasp the whole system at first sight. This continuous learning refines their ideas, and establishes a dialog with the developers, slowly building a vocabulary of classes, tables, structures and ideas. Since they operate as junctions of knowledge, they see when a programmer is in need for a piece of code that could be reused from another's existing code base. They know when they need to put two programmers together. Slowly, they would establish enough common language with their programmers so they could inject their own ideas into the code, when the programmers agrees and understands.

Some may say that there is always some things that would never be achieved. That there are so many obstacles - ego, time, knowledge, so that all we do as architects is just trying to optimize. That perhaps we simply have to learn to live with the guilt that no one, no project, and no architect, is perfect.

But this will not lead us anywhere. Maybe the real answer is that we have to recognize that our main task as architects is explaining. Then, we would invest the time and thought to find the right approach to the sensitive programmer, the right time to experiment coding solutions, and the right words to explain it all.

On XAML, Flash and artistic developers

2007-07-07T18:26:00.000-07:00

My friend and I are discussing these days the new XAML technology from Microsoft. Everyone is very excited about the new technologies from Microsoft, and when those technologies offer an eye-candy, a wow, who can resist it? One of the reasons that I like operating in the software industry field is that sometimes, just sometimes, new technologies appear to be really new. I mean, who cared about the SQL Server 2000 or 2005 or IIS 5.0 or whatever? Zzzzzzzzzz. But sometimes you get to feel that things are really going to make a change. Like when the Object Oriented Programming paradigm started, in the early 90s. With a little introspective, you slowly felt how it changes the way you think. So, this XAML thing looks like another new thing, ok, not in the magnitude of a OOP paradigm shift but definitely an industry-changer.

In XAML – and I believe I don’t make headlines here – the UI is separated better from the code by an XML language called XAML, which is used to define the UI. Microsoft is providing also a tool for designers where they can design this UI without the need to open Visual Studio, which is clearly an industry changer since once you mention Visual Studio to a graphic designer you get a polite Uhm and the conversation somehow fades out. Now, at last, for software houses that use the .NET solution stack, it will ease the workflow – on one hand will allow designers to work on the UI separately from the developers, and on the other hand the Blend does integrate well with Visual Studio, for all the programmatic logic that the developers have to fill in behind the UI. So a good old loose decoupling appears to be helpful in this case as well.

By the way, these are no news headlines to Flash-oriented developers. On that side of the industry ocean, there has always been a great tool for UI creation. The only problem is that there, the tool, Creative Studio, had its initial appeal focused on the art designers, and the programmers stay away. It created a feeling in the industry that Flash programmers suck. The truth is, these were not Flash programmers. These were Flash designers that learnt to program a little. Are we going to see a new generation of artistic developers? That could even be worse.

But the other nice thing is the big change in how the graphics are finally perceived. Finally, somewhat catching up with the game industry, Microsoft architects understood that there are other worlds behind windows. Until XAML, a programmer that wanted to show something on the screen would get what was called HDC, a device handle, that would basically give him a window surface to draw on. So the graphic world was comprised of 2D windows, always straight, always parallel. Now, Microsoft finally changed that. You still draw on a surface, yes. But this surface is no longer connected to a window or a screen. It’s an abstract entity. The .NET DLLs take the UI from the XAML and draw it on this surface. Then, they take this surface – with the stuff drawn on it already – and puts it on the screen according to other attributes defined in the XAML – like its size, position, angle and more. So indeed you no longer think of drawing surfaces as windows, but rather as surfaces that could be put anywhere, in any way. And that is, in my mind, the other strength of the whole XAML thing. By the way, the whole Vista Operating System is based on this Presentation Foundation, and that’s why they could so easily come with the 3D Alt-tab effect (yes, I know its Windows-tab, but let’s just call it the way we know).

So I told all that and more to my friend. I did get the polite Uhm and conversation fadeout but he had to go somewhere. He really did.