My team at work is having a team outing tomorrow, where we will all leave at lunch to go eat and bond over a round of disc golf. Typical team building exercise, I’m sure. Except we’re engineers. A quick conversation about where we should eat for lunch led to a list of about 10 possibilities and an hour long conversation about the best algorithms for properly ranking choices over a small vote set.
I was thinking back today and I remembered something totally awesome from my childhood. And by childhood I mean high school.
When I was a freshman and sophomore, I volunteered for the summer science camp held at our middle school. It was some of the most fun I had in high school. We had various ages, and though my memory fails me even at a young age, we were dealing with elementary school students. The camp consisted of a week of daily science themes, where the science teachers from the school system would each lead a class of kids in a science experiment relating to the day’s theme. Space, Materials and Matter, Earth sciences; all presented in age appropriate ways with several experiments to teach basic principles in each topic. Each class lasted about an hour, which is good because attention spans at that age are not sizable.
As a student volunteer, my responsibilities included helping run lunch time, snack time and helping with whatever experiments happened to be the most involved that day, needing the most hands on help for the kids. Until the last day.
The last day of the camp was Forensics Day. All the student volunteers were assigned a role in a crime mystery. This usually involved the theft of another character’s beloved toy or stuffed animal. The kids would all be gathered in the morning to hear the bad news and that they would be tasked with helping catch the thief! All the classes that day revolved around different forensic methods they could use to identify the thief, be it via fingerprint found at the scene, footprints in the mud, or depositions and alibis. At the end of the day, we’d all gather again and the kids would get to question us with their new knowledge and evidence and eventually the culprit would be brought to justice.
I just checked the school website from my home town. Looks like they’ve expanded their summer camp program. They even have two Engineer camps! Maybe all hope isn’t lost for the future generations.
Maybe.
I like new technologies as much as the next software engineer, but I have my limits. Maybe I’m just more conservative than some of my colleagues. Maybe I’m a Luddite. Maybe I’m too cautious. But some things drive me crazy.
It starts rather innocently. You have a new project. You have an upgrade to an existing system. Your team needs to implement a process solution to support your developers. All are perfectly legitimate projects and all can be solved by any number of platforms and systems.
However, this is where some developers get overzealous because they remember that one cool thing that they read about in the tech magazine that just came out in pre-beta that has the potential to be the perfect solution for all the ails this project seeks to salve. The only problem is that these developers have never used this platform/system before. No problem, we’re all smart guys. We can teach ourselves along the way, right?
Right?
I’ve seen this a few times now in my professional career. I thought we’d leave this behind in college as a industry, but I guess not. And I’m willing to bet that you’ve seen it as well where you work. And it invariably causes more problems than it solves. Sure you may end up with a solution to your original problem, but now you have a much worse problem: you have a monstrosity of a system that operates with all the stability of a tight rope walker that you and your team have to support for the next x number of years.
I did this in school, sure. I was still learning. Having an actual project presented to me was a great excuse to try to learn something new. Of course, that’s the point of college; to learn. Most of the time, professors didn’t care so much about the functionality of the overall project, but were looking to see if you were understanding the concepts of the lessons. Sure, your project crashes if you click that one button with a property set to null, but your sorting algorithm was spot on. Bravo. And you learned some new platform/paradigm/system along the way. Double bravo.
But businesses need things to work. That’s why they hired you. Your place of employ is not in the business helping you learn the latest thing. Your place of employ is in the business of making widgits, selling widgits, or servicing someone who does one or the other. You owe it to your employer to give them the best solution to their problem that you can, which means working with that which you’re familiar.
Don’t fall prey to the marketing gimmicks and hype surrounding whatever the new thing happens to be. It may be great, but not if you use it to create a terrible mound of code that only barely does what it is supposed to do.
I’m not a crank, a luddite, or a developer particularly stuck in his ways. OK, so I might need a little more convincing about the latest and greatest than some of my peers, but I’m definitely not anti-progress.
You should absolutely continue to invest in yourself and your skill set. You should push yourself and train and be a good citizen in the universe of computer science. In fact, I feel that employers should invest in the further training of engineers.
Just don’t do it on the job. First projects are learning experiences. First projects exist for you to learn what to do, and more importantly, what not to do. That means you’re going to write some bad code. That is OK.
As long as you keep it to yourself.
This gallery contains 6 photos.
So I wanted to share some photos I took of my dog today. I recently had the invisible fence that came with my house checked out, turned on, and tuned up. After some training with the dog, she’s starting to … Continue reading
Title says it all. I’m stuck at home home today and tomorrow with an achy body and, as of right now, a fever of 100.7. I’d much rather be at work and spend this sick time on not being sick. Oh well, it is what it is, I suppose. Maybe I can catch something good on Netflix.
There is a lot of hupla (and rightly so) surrounding parallel computing today. CPUs aren’t getting much faster in terms of clock speeds these days and any more its all about the cloud, man. Distributed computing has been a thing in academic circles for quite a while. Much like the Internet, which also started in the world of academia and made its way to the general populace, distributing computing is in its public infancy with projects such as Folding@home and SETI@home, and more recently the computing as a service projects of Amazon and Microsoft. But we’re already seeing a creep into the homes of the average Joe Blow. Multicored GPUs are pretty common now. The GeForce GTX 680 from NVIDIA, for example, has 1536 CUDA cores.
My biggest beef with the state of things right now is that the job of the graphics chip to do heavy processing, even when what you’re processing has nothing to do with graphics. I think its about time that we move away from the current paradigm of a single CPU chip with anywhere between 1 and 16 cores and a GPU with a gazillion cores to a set up more applicable to what is certainly coming in the future. Namely, we need a cluster of general purpose cores specifically for general processing. With the prevalence of data processing in today’s applications, even for local applications, I feel its time to let the GPU do its thing and have a separate cluster for everything else. If for nothing else than allowing games to fully parallelize both graphics specific tasks and game or AI tasks.
Good thing that Microsoft is moving us (the business world, at least) in this direction with the advent of new asynchronous language features in C# 5.0. Really, this is going to have to happen. We need to integrate threading and parallel concepts into the languages we use because in the near future the ability to write a good parallel algorithm is going to be just as important to a developer as the ability to write a good for loop.
I had just one class in college which emphasized the concepts of threading, deadlocking and parallel algorithms, and that was in a class on operating systems. I hope things are changing in computer science curricula, because starting developers out with, and staying with, single threaded C and C++ applications won’t be sufficient in the future.
I’m just sayin’, its time to get on the parallel train. Choo choo.
After having laid dormant for so long, CS-MIC has risen once again. Years in development, and months since the last update, CS-MIC is back with upgrades and improvements, both in the code library and in the project structure. Head on over to find out what the latest and greatest is all about.
As a professional software developer, my job is to plan, design and implement software solutions to problems. This involves learning about a business problem, the process that is followed, and designing software that makes this process easier or even automatic. Almost all software is written this way and for this purpose. And most developers do a good job of fulfilling goal #1: make it work. However, I’ve noticed a disturbing lack of attention to what should be goal #2: presentation.
Of course, I’m generalizing. I’m lumping some facets of software into making it work and lumping others into presentation. Let’s, for the purpose of this post, go with the following definitions:
In business, function is everything. This is unfortunate, because form matters, both in interface and structural design.
The most obvious, and public, example of form is in your user interface. The control layout, the data grouping, the flow all coalesce into an important feature of your software. A good user interface can greatly improve the productivity of your users, which is, by the way, the reason you exist as a software developer. Conversely, a poor user interface can harm your users productivity. Taking advantage of your preferred development platform’s (which happens to be C# .NET for me) built in functionality can do wonders for your project; it makes the difference between good software and great software. Simple implementations, such as setting intuitive tab stopping on forms or writing clear and specific error messages, make the lives of your users easier.
Most overlooked, and most difficult to solve, is the problem presented by performance that is acceptable. I see code all the time that is never optimized because at some point an algorithmic solution to a problem was found, and even though it isn’t as fast as it could be, it just works. Sometimes this code exists for legitimate reasons. There are times when the simplest code is not as performant as a convoluted equivalent. This is rarely the case though. In my experience, this has usually been used as an excuse by people not interested in spending the time to correctly structure or comment their code. I distinctly remember in college being told to always go with the simplest coding option because memory and processing time is so cheap now. To this I say bah. This is the kind of thinking I see that has led to database calls, file processing a myriad of other processes taking minutes to complete when they could have been run in seconds.
You may ask why this matters. It matters because the interaction between the user and the software is a very critical determinant for the success of your software. Your users probably won’t be able to tell you that a database call is just a little laggy if they have never made a database call themselves. Your users won’t be able to let you know that you should change the tab stop settings on a form or that the work flow for that thing they do over and over all day could be tweaked or automated. Users aren’t, for the most part, trained to analyze their workflow in this manner to watch for inefficiencies. Hell, you’re lucky if you find out about legitimate bugs before entire business processes spring up to workaround it until you’re left with a breaking change in the fix for the original bug.
What your users will know is when you get it right. The right interface and optimization of backend processes can increase productivity. More importantly, perhaps, is the increase in user morale that the right balance can bring. Happy users get more done. Happy users don’t dread working with your software. Happy users make your life easier.
The perfect example of good and bad presentation, I feel, can be seen in the current mobile OS wars between Apple and Google. I think most people, despite any perceived shortcomings of iOS in its feature set, one must admit that the presentation and performance of existing features is top notch. The iPhone took off the way it did because everything about iOS was a piece of art. Apple has sacrificed numbers and features in exchange for perfect presentation.
Google, on the other hand, seems to side more with a rich feature set, but with less desirable presentation. Android feels like it’s been designed with the average engineer in mind, whereas iOS feels like it has been designed for the average not engineer. This is one of the reasons I’m against owning a phone running Android. I have yet to see a phone running Android behave as intuitively and fluidly as an iPhone. For example, the scrolling animation is always jittery. When I slide a control on the iPhone, I feel like I’m interacting with a physical entity. When I slide a control on an Android phone, I feel like I’m sending a command to a computer and it is having a hard time keeping up with me. That’s a purchase (or even a series of purchases) that Google and the respective cellular carriers are losing out on because of a lack of attention to presentation.
These things matter. A lot. There are costs in productivity, morale, and sales associated with presentation. Don’t lose out. If nothing else, just consider it another engineering challenge: optimizing your users.
Standing in the bathroom waiting for the tornado warning to pass. I think the people in the hall are missing the point of taking shelter. Doesn’t matter if you stand outside in the hall if the hall is wide open and has two large open rooms on either side with window walls.
1/2 inch glass isn’t going to stop that tree hurtling toward you. Just sayin’.
So for two days I’ve been beating my head against the wall at work. The task was simple enough:
Easy enough, right? Not so much.
I started this task like I would any other data centered task. I started with a script to get the data I needed. Easy-peasy. I had to gather data from several different tables and do some filtering based on data present in other tables. Nothing too complicated. I ended up with something along the lines of:
Didn’t take long to crank out this script once I worked out all the requirements and figured out which buckets to dump the preliminary data into. So now on to the easy part. All I have to do is run my query in SSIS and dump it to a text file.
So easy.
So I’m new to SSIS (though I’ve learned more about it in the last two days than I’d have liked). I found the Data Flow control and thought I would be done by lunch. I thew in an OLE DB source and pasted in my magic query. Of course this angered SSIS. I was quickly told in no uncertain terms: F off.
Problem: Turns out you cannot use DECLARE or CREATE syntax inside of a sql command for an OLE DB source.
So I couldn’t use my script directly. Fine. I’ll make it a stored procedure and return my data set. I set up the procedure and we’re in business. My somewhat complicated query runs on our test box in about two minutes. So I plug this into the OLE DB source via an EXEC in the SQL command. This proceeds to destroy Visual Studio. I give up on the process recovering after 15 minutes.
Problem: SSIS, in the interest of being easy makes you rely on magic to discover metadata about your queries to automagically populate column and transform information for you. Unfortunately, part of this magic involves adding a preamble to your query to pull metadata and rows as a preview. This can lead to some terrible query plans for the metadata retrieval call.
The thought occurred to me that if SSIS was searching for metadata on the return results of the stored procedure, I’d bypass this magic by just turning the stored procedure into a function. After trouble compiling the TSQL as a function, I quickly realize that temp tables are not allowed. I have to use table variables due to the data integrity contracts for SQL Server user defined functions. No problem. Quick fix. Back to the OLE DB Source and plug in a query against my function, passing in the parameter in fnMyFunction( ? ) form. SSIS laughs in my face.
Problem: SSIS is perfectly capable of parsing your query, as long as you don’t try to pass a parameter to a function. Oh, so your parser understands ( ‘hard coded value’ ) but not ( ? ). You’re a dick, SSIS.
I was now on a mission. It was time to go rogue and break some rules. So I broke out of the data flow and into the control flow. I added a script and added a CREATE/DROP statement for a temporary table (read: not a temp table or variable) that was actually going into the schema. I then selected the returned result set from the stored procedure version of the script into this table. I then piped into the data flow and just ran a select all from this table. SSIS was perfectly happy with this since I turned off design time validation on the OLE DB source. I funneled this into my flat file, then piped from the data control to another script that dropped the table before finishing execution.
While in the middle of this giant mess, my boss, while laughing at me for having the bad fortune to get this hellacious bug, told me that I was actually lucky that I was getting to work with SSIS rather than DTS. He then launched into a list of all the ways SSIS was superior to DTS.
That’s all fine and dandy, but if I’m looking at two cars, it doesn’t matter if one car has a 400 horse power engine, leather interior, and jetpack wings if I can’t start either one and get from point A to B.
Who knows. Maybe SSIS will grow on me when I have some simpler queries to run. You didn’t make a good impression, though, SSIS. I’ve got my eyes on you.