Blog
Heartbleed and code opacity
There is an old open source saying that "Many eyes make all bugs shallow". So what happened with the now infamous heartbleed bug? Supposedly, open source code is less prone to these kind of security breaches because so many eyes can look at it. In addition, open source is supposed to have better fix properties because fixes can be turned around and disseminated quickly. However, something happened with heartbleed. One issue is that most eyes are shallow, not deep. If code is opaque (as is the case for Open SSL) than it doesn't matter how many eyes are on it - bugs can be hiding in the submerged depths where only the deepest eyes are likely to stumble upon them. And it is likely that the number of deep eyes looking at the code is still as small as is usual in any other software, closed or open. The second issue is that the decentralized nature of open source movement actually harms as much as helps with dissemination. Commercial software, with its lists of registered users at least has a starting point to make issues known and patches distributed. Open source projects tend to have much more chaotic and imprecise means of the same. No one, commercial nor open source, can be sure that software will be patched - but at least commercial software has a starting point. So, what can the open source community learn from this issue? First, make sure that the code is written in the clearest possible way. Make sure that as many eyes can peer into its depths without obscuring murk. It may hurt performance, especially with ubiquitous software like Open SSL, but the alternative, as we have seen, sucks. An old wise man once said that "Premature optimization is the root of all evil." Although this is true, code that needs to be secure cannot afford any optimization - the code must be lucid enough to be easily understood and checked, lest hard-to-detect security defects sneak in. God knows you probably shouldn't be wrapping OS calls (the putative source of this defect). Second, for code as ubiquitous and as important for security as this code is, independent review by people qualified to do so is vital. Most projects have only people related to the project reviewing the code. This is probably fine for the latest Zul Zebra Zaga game, or other kind of fart app, but not so good for a piece of code upon which most of the current internet's security rests. Third, it is almost certain that the crucial parts of the Open SSL code, had it been written in a simple intelligible form would have been small enough to be formally verified. And, again, given the criticality of this code to secure operation of the internet, it should have been done. Do all of these ideas have costs? Yes. But so do bugs that have as high an impact as heartbleed appears to. We've known how to not come a cropper when the stakes are high for at least the past thirty years. The fact that these kinds of issues are still arriving (and in such crucial code) is an embarrassment to the whole industry. |
Software Engineering: 8 - Align process with your environment
You wouldn't wear a parka in July would you? If you were in Antarctica, you would, because there at that time of year, it's winter... Plus, let's be clear - it's Antarctica, and the average temperatures, even at the height of Antarctic summer, don't get above zero. Your environment matters. A lot. It matters when designing processes for your organization, as well. A good process must be aligned with four important features of your professional environment - the market into which you're selling your products, the nature of the products themselves, your organization, and the skills and abilities of the teams within it. Let's look at market influences first. If my market is consumer-oriented, dominated by social media and moving at lightning speed, I don't want to wait for a traditional process to "get it right". I'm going to be putting my best agile practices into place and releasing every two weeks. On the other hand, if I'm releasing into a more static enterprise environment, I probably don't need frequent releases (in fact, my rather conservative IT customers are probably unhappy if they have a new version of the product more than once a year). As such, a traditional process with decent change control mechanisms might bring better results with less uncertainty on which features will be delivered. It's the same with product alignment - a web-based product can have more frequent releases than one that needs to be updated on every client's system. The process for the first product should target a minimal time to release, whereas this factor is less important in the process used to develop the latter. The process for the second should take the essential difficulty of the upgrade process into account and, if possible, use this essential issue to its advantage (not that one shouldn't make the upgrade process as simple as possible in any case). If my product needs language translation, I probably (for cost reasons) don't want to have new features translated for each two-week release. I'd cut the releases to once each quarter (even if my internal releases were on a two-week cadence) and integrate a sprint/iteration/whatever to have the product translated, rechecked, and released. Similarly, organization counts. If I have a company that boxes both time and features and that doesn't keep teams stable, my process should look different from one that boxes only time and has stable teams. If my marketing department needs a half a year to plan a product launch, it doesn't matter much if my team can release on two week's notice and my process should take this into account. It's the same with the skills and abilities of the team. An inexperienced team will need a lot more design and code reviews in its definition of done than an experienced team - maybe even more QA cycles. And your process needs to take this into account. That being said, if your process isn't aligned with these four facets of your environment, it doesn't mean there's a total disaster brewing. A good team can make almost any process work (For varying values of "work". N.B.: The converse is not true - a bad team can make hash even if you have a good process in place). A poor match between environment and process usually just means that it will be more work to get the results you want (And who wants that?). The other thing to realize is that any standardized process you put into place will vary in its alignment from release-to-release, product-to-product, team-to-team, etc. Markets change; products change; teams change. As such, one must be ready to adapt one's process. And, again, failure to do so won't necessarily bring disaster, just more work. So, when putting a process into place consider carefully the four environmental attributes listed above - market, product, organization, and team. If you don't you might find yourself in a situation analogous to being at the bottom of the world in shorts, sandals, and a tee shirt. And nobody wants that. |
Getting ready to record the third This, Not This! EP
The headline says it all. The band is looking for a more organic feel with all of us recording basic tracks at the same time. We'll be recording this upstairs in my house, again, trying for a more organic feel. In all of this, I'm taking the philosophy that simpler is better and that's reflected in way I'm miking the drums. I started by using my bedroom for the drums - it's the room with the best sound in the house. It's basic sizes are not multiples giving it complex room modes and it also a gabled ceiling to add reverberant space and further reduce room modes. It also has a bed and some furniture in it for reflection and absorption. Finally, it opens into the hallway that opens onto the bottom floor, giving it a huge ambiance, if needed. We started miking by getting a pretty good sound for the drums in the overheads (a pair of matched Mike Joly modded MXL 990s). Next, a Shure Beta 52 was placed to reinforce the kick and a couple of packing blankets tossed over that to reduce bleed. Then, I added a Shure SM61 (a venerable old dynamic vocal mike with a nice high-end sizzle) to reinforce the snare. I balanced the mix and put up an Audio Technica AT3035 as a room mike and added that in for some more ambiance (since it's going to be smashed to hell when I mix, any LDC mike would work here, but the AT3035 is familiar to me). Until this setup doesn't give us what we need, this will be the default. In the past, I've used another mike on the hi-hat, but that always seemed to lead to nothing but snare bleed and heartache, so I'm not using one this time. Another omission? Individual tom mikes. In general, I find that using these only added to the ringiness of the toms (not my favorite sound) and that boosting the overheads in the mix during the fills works just as well. This simpler miking technique should also reduce (if not completely obviate) phase issues, too. So, simple, simple, simple and it sounds pretty good. The guitars will be recorded using Cascade FatHead II ribbon mikes. The engineer on our first album used them and I had good luck with them on the last album, as well. They just seem to sound good for this purpose. Gary will go in the laundry room, Phil in the guest bedroom. The bass will be DI'ed from the control room, as I can muck about with it as much as I need with my standard plugins. As for vocals, Debbie will be using an Ear Trumpet Labs Edwina in the guest bathroom. This is an LDC built by a local microphone maker named Phil Graham, optimized for live use. Intention notwithstanding, it sounded very good for Debbie's vocals on the last album and it has a certain mojo in it's look that seems to make people comfortable while recording. Pretty straightforward setup here - pop screen in front, an absorbing screen in back to try to tame some room reflections. You notice that I'm not mentioning preamps here. They say that clean is the new color. If I really need color, I'll put it in by re-amping, not preamping. I'm keeping everything clean by using solid state preamps - a combination an M-Audio Octane, a PreSonus Digimax, and three Symetrix 202s. I may break down here and use a UA 610 as a preamp for Debbie's vocals (Yes, clean is the new color, but we can go too far with everything, can't we?). I've also been taking time to set up separate headphone mixes for all involved. The four channels for Dave, Gary, Phil, and myself will go through my Furman HDS, while Debbie's channel will be a separate full mix (controlled by the recordist - i.e., me) fed into a discrete headphone amp. My upstairs looks like the cable fairy came to visit. All tidied and zip-tied around the edges of the rooms, but still a quite unique look. Have I mentioned that I have a very tolerant and loving wife? I do. So, on to recording... Of course, the main battle in recording an album like this is getting everyone together at the same time to do it, so I don't see it starting until the first week of August. The songs that we will be recording include Goodbyes, Takedown (not to be confused with Takeaway), Proof of Life, and a new one with the working title Hardhitter. In the mean time, I'll probably have Dave do some drumming for some tracks on my solo album. |
Software Engineering: 7 - Beware the purple squirrel
In HR circles, "purple squirrel" is a term that is used to describe the candidate who perfectly fits a multifaceted job description. In theory, the purple squirrel would be up to speed in an instant, providing immediate value in all facets of the role. However, there are many problems with hunting the purple squirrel... First, you will take a very long time finding such an animal. Months will be spent on the search, interviewing a plethora of rodents just to find that the squirrels in question are run-of-the-mill brown squirrels (or even worse, rats with a tail hairpiece) holding exaggerated resumes. This effort will distract you and your team from other tasks that they might be doing, putting you farther behind. Next, assume you do find a somewhat purplish-tinted rodent. Your team's desire for the perfect purple squirrel will engender endless debate over whether the shade of purple is too blue or too red or whether the rodent in question is a tree squirrel or a ground squirrel and whether that makes any difference. Decisions on whether to use this rodent will take too long and, quite rightly, many purple squirrels will escape your grasp because you grab at them too late. And, still, all the while, your position will go unfilled and things will fall farther and farther behind. When you can find a purple squirrel that your team agrees on, you will find that purple squirrels generally understand their rarity and will demand top compensation for their uniqueness. Even if you can find the purple squirrel, you may not have the budget to pay for them. If you do, you can assume that there are others out there also looking for your purple squirrel, ready to whisk him or her away for a few more peanuts. Now, assume you can and do pay for a purple squirrel. Bring it into your environment and you'll notice that their shade isn't quite the shade you needed anyway - there are enough differences in your environment that you end up training the squirrel to be your shade of purple anyway - the squirrel will not provide the immediate impact that you expect. Plus, to keep the purple squirrel effective, you'll have to feed it a special diet of training to keep its unique skills in shape. If you don't do this, you may find your purple squirrel taking on a decidedly brown shade after a while. Finally, three months from now, when the environment changes and you need a green squirrel instead of a purple one, you'll find that your purple squirrel, having invested many years in eating the proper food and finding the proper environments to allow him to survive as a purple squirrel, will object to your attempt to dye him green. More insidiously, if you attempt to modify your environment towards a different color, you will find the purple squirrel more likely to sabotage than to support the change. Seriously, you'd be better off hiring an ordinary brown squirrel who has shown a willingness to be dyed whatever shade of color you need. In reality, many job descriptions are written in too detailed a fashion, not examining how work could be reassigned among other team members or assuming a candidate needs immediate capabilities in tasks that need to be done sporadically. Employees with unique skill sets decrease your "bus factor". And the managers who look for people with all of these qualifications fail because they either don't want to take the time to train or because they don't recognize the difference between satisficing and optimizing. Ultimately, their projects fall behind or fail, too. You can avoid purple squirrel hunts by performing regular skills assessment for your team and making cross-training a priority. The latter will also improve team-member engagement by increasing their sense of mastery. If you do end up with a purple squirrel on your team, add one more requirement to your list - that your purple squirrel can teach other squirrels how to be sort of purple. By doing this, you'll reduce the need to hire a purple squirrel the next time. So, structure your team to reduce the need for purple squirrels. In the end, you'll find that a group of happy, harmonious, flexible, and (most importantly) high-quality but somewhat ordinary rodents will probably fulfill your needs just fine. |
Software Engineering: 6 - Observe the 10-60-30 rule
"Ninety percent of everything is crap." -- Theodore Sturgeon Ted Sturgeon was talking about writing when he came up with his law and I think he was correct in setting his cut point for excellence at this point. However, when you talk most things - and specifically things like software - I think that his law is overly critical. Ten percent is probably the proper cut-point between excellence or genius or amazing or whatever other superlative adjective you want to use and everything else, and you probably need to meet that high bar in writing or any other artistic endeavor to be seen as original and making an impact. However, just because something isn't brilliant or original doesn't mean it isn't serviceable or useful - which is the bar we set in most engineering fields. As such, I propose an updated form of Sturgeons Law for engineering that I call the 10-60-30 Rule: 10% of everything is excellent, 60% of everything is useful and/or serviceable, and 30% of everything is crap.* Original and brilliant need little explanation. However, there are many parts of our systems that don't require great genius - just good solid code that does what it's supposed to. This good code comprises 60% of most systems (and I would presume 60% of writing, art, and music creations). The final 30% is bad - it needs rework or excision and replacement to move it to the categories above. One other aspect of this rule? The corollary that, even after refactoring and removal of the bad code, the ratios above are still descriptive to the body of code - it's just that you've raised the bar a bit. And this is why code maintenance is so important - it's hard to make the code better by improving either the good or excellent code. But by improving the bad code (which is easier to do), you've improved the quality of the whole code base, and (almost certainly) uncovered new issues in the good portions of the code that can now be improved. So watch for that 30% of bad code. It's a high enough percentage that you almost certainly know where it's at. Then take the initiative to fix it. It won't get rid of bad code (remember, once that this bad code is fixed, another 30% becomes the worst in the code base), but it will raise the bar on your code's quality. And the people who find the code that you have left will be grateful. *Actually, you can break down the larger categories further: 10% is excellent, 20% is really quite nice, 40% is good, solid work, 20% is bad but salvageable, and 10% reeks and should be replaced. However, "10-20-40-20-10 Rule" doesn't roll off the tongue as nicely as "10-60-30 Rule" does and, since what you do with the two classes of good and bad code are the same - leave alone and rework ruthlessly, respectively - the latter seems to be better. True aficionados of software numerology may have fun with the fuller definition. |
CD Release Party
The latest CD from This, Not This!, Waiting for the World, has been released! To celebrate, we're holding a CD Release Party at the Lounge of the Hawthorne Theater in Portland at 9 pm on 29 March. Our old friend Mike Ailes will be there with his band, Aina Haina, to open for us. It's going to be a great show, with lots of songs from both our first and the new CD. If you'd like to have your own copy of the new CD (or the first one), they'll be on sale there for $5. Of course, you can also buy it from iTunes or from CDBaby. I'm really proud of this album, as it's the first I've recorded and mixed for the band. And, of course, come out to see us and Aina Haina play! |
New Songs and Upcoming Gigs...
I'm doing final touch-up on the mixing and mastering for the next single from This, Not This! - a song I wrote called Dimensions. It's been a favorite in our live shows and I'm looking forward to having it finished and up on iTunes and Amazon Music sometime next week. Frankly, I did a car check on my mastering for the single and I think it actually sounds better to me than the songs on the EP which we sent out for "professional" mastering (of course, I'm biased). This one really rocks, so take a listen at the streaming "pre-release" on the This, Not This! website. Look for it when it's out there, buy it, and enjoy. Next up? Mixing one of Phil's songs - Touch of Magic - and recording harmony parts on another one of mine - Waiting for the World. Dave Wakeling and his wife, Candace Schimp, and I will have a three hour acoustic show at Urban Decanter, a wine shop and restaurant at 2030 Main Street, Forest Grove at 7 pm on October 20 (this coming Saturday). The material will be mainly Dave's, with some covers thrown in, but I'll sing a few of my songs, too, including a couple that have never been performed live before - Icarus and Stupid Genius. I'll be playing bass and singing harmony, for the most part, but will also help Dave out on guitar on some songs. Dave, Candee, and I always have a great time performing together - wonderful songs and harmonies and you might get to hear Dave call me a wimp, live, for not wanting to play certain songs so much. A good time will be had by all, so get there and listen. On December 7, Dave and I will be playing with This, Not This! (our electric band) at the Mt. Tabor Theater at 4811 Southeast Hawthorne Boulevard in Portland. As usual, about half the songs will be mine and I'll be playing bass and singing backup. Appearing with us will be Aina Haina and Ill Lucid Onset. Aina Haina features our old buddy Mike Ailes, who has opened for us before. This is, however, the first time we get to hear him with his band - we think it will be really great. We met Ill Lucid Onset when we played the Clark County Fair in August - they had a really good show and some great songs. We're looking forward to playing with all of these guys. Also, if you haven't heard them yet, check out Dusty Santamaria and his band, The Singing Knives. Their online material doesn't do them justice - Dusty is a riveting performer and his performances shouldn't be missed. Sherry and I have been trying to get This, Not This! and Dusty together for a show for... well, for a long time. We hope to have a show with them soon (probably sometime early next year) - negotiations with Dusty and various venues are underway. Keep checking back to the This, Not This! website for more information. Or better yet, go there and join our mailing list, after which you can be notified of all of our upcoming stuff! See you at a show soon! |
Focal Point Mixing
Joe Gilder over at Home Studio Corner has a some good tips about balancing levels in songs. I'd add one other little bit of advice with respect to overall dynamics: One way to judge whether a mix's levels are right is by listening to the relative levels of "focal points" in the song. What's a focal point? At any point in time, one of the parts is usually the focus of the song, whether it's the vocal or a lead or a drum fill. Use automation to keep these focal points even in perceived volume (+/- a few dB) and make them louder than the rest of the elements in the mix (they are, after all, the focus at that time). Because your final bus compressor will key off the loudest element in the mix (OK, it's a bit more complicated than that, but I don't want to go into it now), the compression will be more even and the overall mix will have fewer compression artifacts. As an added benefit, it will make the mastering process easier. I can hear the complaining now - "But, Frank, I've just reduced the drum volume to put them in the back of the soundfield. Now you're telling me to turn up fills?" Yes. I'm telling you to bring the drums forward on fills, assuming the fill happens to be the focal point at that time. Drummers tend to hit fills harder than normal rhythm strokes, so we're used to hearing them get louder - it works psycho-acoustically and a bit more volume won't break the fourth wall. Same way with that killer bass run. Or that interstitial guitar lick. You don't even have to increase their volume that much - you can also use automation to slap an EQ on the focal points to boost them in the presence band (3-6 KHz, season to taste) or turn down their reverb a bit. It will make them stand out without as much overall level change - we're talking psycho-acoustics here, not numerical equality. Getting focal point levels right will go a long way towards making your mix even. And an even mix is one of the hallmarks of a good mix. |
Software Engineering: 5 - Make the process your own
There is no "true" chili. Those of us who love this spicy concoction understand this. For every region, there are different flavors. For every recipe you know, there will be a dozen others you don't. And almost all of them taste good, in the right place and at the right time. You can find some basic commonalities - a chili often has onions and some sort of pepper, for instance; but for every "rule", there seems to be a recipe that shows that the exception can be just as tasty. Chili aficionados have responded to this uncertainty by generating a plethora of variations. And, even if you hand a chili cook a recipe, he or she will usually modify it somehow. It's the same with software development processes. We are handed cookbooks full of procedures and rules. Sometimes these fit. Sometimes they don't. The reasons why things might not fit come from many different directions - the nature of the market and its timing; the nature of the organization; the nature of the product, its delivery mechanism, and the tools used to construct it; the nature of the team - all of these can impact whether or not a particular part of a process is effective. More importantly, sometimes these forces can make a portion of a process worse than ineffective - it might take more work than it saves, creating more work for your team without providing additional benefit. As such, it's clear that processes often need to be modified. In addition to removing inefficient practices, processes can be improved by adding or replacing procedures. I've seen traditional processes improved by importing steps from the agile world - using TDD practices or delivering frequent intermediate builds can often help a traditional team achieve better quality. Depending on the product and frequency of market or requirement changes, an agile methodology can be improved by formalizing and adding a bit more structure. The main issue is to always be aware of what's working and what's not working and fixing the thing's that aren't. Occasionally, you'll want to change processes anyhow, just to boost your team's efficiency via the Hawthorne Effect. Plus, introducing teams to new process practices will help keep them fresh and ready for change if (actually when) your team's environment changes and you need to change your processes in response. So, if you're the leader of a team, don't be afraid to change your processes to make them more effective - make them your own. Keep trying to make your team's development and project management processes more effective. It may take some experimentation to come up with the right mix of process "ingredients", but the payoff will be in the tasting - when your high-quality product arrives on time. |
Why Simbi is my lwa
I made mention on Facebook that I was reading a book on Haitian Vodou and how it was possible that Ogun, being one of the technical lwa might be useful to practitioners of software engineering. But I think, at least for me, now that I have read the book, that the Simbi are a much better match. To see why, let's do a quick description of each... Ogun is the lwa of iron and the forge. He is also a mighty warrior. If you think of a cross between Mars and Vulcan, you will get the idea. So, although Ogun can be used for technological purposes, he is much more suited to designs of rifle barrels and predator drones than run-of-the-mill technology. In addition, Ogun is a very powerful and prominent lwa. Asking Ogun to help with normal computer issues would be like using a tactical nuclear weapon to swat a fly. In contrast, Simbi, the water snake, is able to exist in both salt and fresh water and, as he mediates between these two world, he also mediates between the upper realms and the crossroads. He one who leads souls from the visible to the invisible starting at Legba's crossroad and who leads the invisible to the crossroad to accept the sacrifices of those who are visible - he is the hermetic Mercury. More importantly, he also is the lwa of rain. And, just as the rain brings creation to the parched earth, so does Simba bring the creative element in his wake. In addition, just as water heals the earth, so is Simbi a healer whose knowledge of the herbal world allows his followers to heal others. As the lwa of communication at the speed of light, he is very handy with computers and other telecommunications devices (which are often in need of repair). Finally, he is much more tractable than Ogun and more useful to ask for help in more common technology matters. In addition to being attracted to Simbi's aspect of healing the world, I was also born under a water sign, further connecting me. His water aspect explains my love of long baths and showers (the reason my house has two water heaters), and his serpent nature explains why Kundalini was the only form of Yoga that ever resonated with me. And, I think this morning he helped me get my audio interface working again. I had an insight that maybe I was plugging the unit into my Firewire card instead of the PCI interface card. I looked at the back of my computer system and at the connection and It all looked OK. I deracked the interface unit to get its serial number to register it so I could submit a trouble ticket. As I re-racked the unit, something continued to nag at me. As I looked at the back of the computer system one more time, I realized that the first time I checked, I was mistaking the audio interface's PCI card as a video card because of the connector at its bottom that looked like a VGA socket. Connecting the audio interface to this card made the system work - the initial insight was correct. Now I can't say for sure whether or not this insight came from Simbi, but just to be sure, I left him a cup of cola under a nearby tree (I didn't have any black coffee or rum on me, and he said cola was fine). |
1-10 of 48