Kerry Osborne's Oracle Blog

Well our new Exadata showed up this week. We had a pretty nice lab environment already. A bunch of Dell’s, some IBM’s, several Sun’s. We have a couple of EMC Sans as well (we actually threw away a whole EMC rack to make room for the Exadata). And of course we have every version of Oracle from 8i to 11gR2. It’s a good learning environment. It also let’s us try things when clients have a specific set of versions that we want to mimic. So now we have an Exadata V2 as well. We’ve had the delivery date on the calendar for several weeks. For some reason it reminded of the Weird Al Yankovik song Weasel Stomping Day.

It’s probably a sad reflection on how geeky we are that everyone is running around all excited like it’s Christmas or something.

Here’s a few pictures:

It’s a really fast machine by the way. In fact, we had trouble keeping up with it from the moment we got it off the truck.

Really fast, and slippery. Well in a couple of days we can actually turn it on (we’re supposed to let it acclimate to our environment). On Friday afternoon we’re going to have a happy hour to celebrate our newest edition. Wade calls it a sip and see. We’ll probably take a few pictures of ourselves with the little bundle of joy and sing a chorus of a festive Weird Al song, or maybe two. Come on by if you’re in the neighborhood!

I went to the Dallas 100 awards banquet last night at the Morton Meyerson Symphony Center. The Dallas 100 is an annual award for the fastest growing privately held companies that are based in the Dallas / Fort Worth area. Enkitec was number 81 this year – not that great, but still not too bad considering the state of the economy over the last couple of years (we were 42nd last year by the way).

Sam Wyly was the guest speaker at the presentation. He didn’t actually give a speech, but rather gave an interview which was interesting. He’s best known as the founder of Michael’s Stores which sold for around $4,000,000,000 a few years back.

As usual, there was a dinner after the presentation. And as usual it was some kind of meat with asparagus. I guess asparagus must be one of those things that can be cooked a little bit ahead of time or kept warm without too much trouble because it seems to be standard fare at these types of events. Anyway, it’s always nice to have a dinner with asparagus because you get reminded of it the next morning. I won’t go into the details here but if you’re interested I highly recommend this article on the Discovery web site. Whew!

I am constantly amazed by how people, myself included, occasionally over exaggerate. It’s pretty common in normal speech but it seems to be almost required in marketing material. Words like Greatest, Biggest, Most Effective, Fastest, Fluffiest, Zestiest, etc… I sometimes wonder how people come up with this stuff. For example, I ran across this video on utube:

Best Guitar Improvisation Ever

Entertaining no doubt, but “Best Guitar Improvisation Ever”???

Who voted on that? Were professionals allowed to compete, like say Mark Knopfler, Eric Clapton, Slash, Stevie Ray Vaughn, Carlos Santana, Buckethead, Jeff Beck, Peter Frampton, Eddie Van Halen, Jimi Hendrix, Roy Clark, Django Reinhardt, Chet Atkins, B.B. King, Jimmy Page, Andres Segovia,  …  ? (sorry if I left off you’re favorite, but you get the idea)

Some of it is shameless self promotion, but I blame most of this stuff on marketing guys. I don’t hold it against them, they have to make a living too. But they do a couple of things that amuse me.

1. They make the previously mentioned exaggerated claims.
2. They make up official sounding statistics to support their exaggerated claims.
3. They write their own quotes (and attribute them to people that they wish had said them).
4. They are obsessed with “look and feel” stuff (“It’s more important to look good, than to feel good”).

Let’s begin with bullet number one (exaggerated claims):

Here’s a couple of outrageous claims I ran across (names altered or withheld to protect the guilty).

• “Joe and Bill are recognized as the two best-selling Oracle Authors in the world.  With combined authorship of over 25 books, Joe & Bill are the two most respected Oracle authorities on the planet.  For the first time ever, Joe & Bill combine their talents in this exceptional handbook.”

• Did you know that there’s a way to burn 3 to 15 pounds of fat PERMANENTLY and SAFELY . . . WHILE YOU SLEEP?! 100% Guaranteed!

• “Because you haven’t just found people who know a little bit about what you’re going through – you’ve found the undisputed heavyweight Oracle experts.”

That last one was bad enough without emphasizing the outrageous claim by using a bold font. And just to show I’m not playing favorites, that one is from the web site of the company I work for!

Continue reading ‘Worlds Greatest Guitarist’ »

I read in the Fort Worth Star Telegram this morning that the Texas legislature had passed a law (Texas House Bill 171) which requires school administrators to use common sense. I guess there has been a problem with this in the past (actually the bill explicitly states that there has been a problem – “School administrators are allowed to consider mitigating factors but sometimes choose not to exercise common sense.”). At issue was strict “no tolerance” policies in some school districts whereby kids were getting sent to alternative schools for infractions as absurd as leaving a baseball bat in the car, leaving fishing tackle (including a knife) in the car, forgetting to take a pocket knife out of a pocket, getting beat up (that’s right – both participants in a fight get the same punishment).

Oddly enough, no one voted against this bill. How could you vote against “common sense”. Can’t you just see the negative political ad next election …

You know the unflattering black and white images, with the voice over saying something like:  “My illustrious opponent voted against Common Sense. Surely you don’t want to elect someone who doesn’t even believe in Common Sense!” …

During dinner, my daughter asked me what I was going to do at work tomorrow. I told her I was going to try to use common sense, even though it wasn’t required by law in my profession. At least it doesn’t yet… But imagine if we could just get a few more organizations in a few more states to start taking this idea seriously. Just imagine what could happen. Reminds me of that bit from Arlo Guthrie’s song – Alice’s Restaurant:

“… And can you, can you imagine fifty people a day,I said fifty people a day walking in singin a bar of Alice’s Restaurant and walking out. And friends they may think it’s a movement. …”

(by the way, if you’ve never heard “Alice’s Restaurant” you should go buy it from iTunes right now – I mean right now – don’t even finish reading this – do it now!)

Anyway, this common sense thing sounds like a great idea that could really catch on. I can think of a few organizations where common sense could really be valuable (a lot of them are in Washington). Although I’m not sure how they would enforce it… But that’s a job for another day.

Recently I’ve been thinking about how often we DON’T fix problems. I mean, we usually make the situation better and often in very short order. But a lot of the time we don’t really fix “the problem”. We just do something to cover it up, or make it less painful. I call it putting Band-Aids on the problem.

The Band-Aid metaphor works on several levels.

1. Band-Aids don’t actually fix the problem, they just cover it up.
2. They aren’t meant to be permanent.
3. And they don’t keep you from stabbing yourself again in another spot.
4. But they do make it less painful.
5. And they keep out germs so they keep some problems from getting worse.
6. And they are cheap (all those boxes in the picture cost less than $20 combined).

There are all kinds of special purpose Band-Aids for special applications. Like the ones that have antibiotics, butterfly bandages for closing big cuts, etc… When it comes to Oracle, my personal favorite Band-Aid is memory. Additional memory covers up a lot of sins. One of the primary design goals of the original Oracle database was to eliminate disk access. This remains a key objective today. You can think of the database as a very sophisticated disk cache. The goal for a lot of systems is to never do real time disk access (with the exception of commit processing). So it stands to reason that memory is a key component (maybe “the” key component) of most Oracle systems. And it’s relatively cheap. There is often a surplus of memory already installed that is just not being used as effectively as it could be. But even if it’s necessary to add an additional memory module to a server, this option is generally very inexpensive when compared to other potential “hardware upgrades”.

Don’t get me wrong, there are a lot of valid reasons for using Band-Aids with Oracle. I know some people that seem adamantly against applying short term fixes instead of addressing the underlying problem. But it seems to me that in certain circumstances (which seem to occur fairly often actually) Oracle Band-Aids are an appropriate response. A few of my favorites reasons for applying them are:

1. Buying time to figure out what’s really going on and how to fix it
2. Buying time until the system is decommissioned or a new version is rolled out
3. Buying time until the developers can get the real fix through change control (and as we all know, sometimes this takes a while)
4. Just decreasing the pain because it’s a packaged app that we can’t change

I do think we should be vigilant about not overdoing it though. Otherwise our databases end up like the guy in the picture below. Pretty messy and sometimes hard to even figure out what the original intent was.

Along the same lines, keep in mind that Band-Aids often leave a lot of left over trash:

Anyway, that’s my rambling for today.  Just for fun, here are a few more pictures of some unusual Band-Aids.

Feel free to let me know what you think about applying Band-Aids to Oracle systems. And what your favorite (or least favorite) Oracle Band-Aids are.

I had an interesting discussion last week with an Oracle DBA that I have a lot of respect for. He surprised me by telling me that he thought development systems should intentionally be built to be slow. That way, the developers would be forced to write efficient code. He was not just stating a philosophical position either. He had actually built some databases with 32M buffer caches (the one I mentioned in this previous post on Buffer Cache Extremes was one of his). He went so far as to say that he thought that developers should be using 286’s for their development systems.

I have argued for several years that development systems should match production as closely as possible. But that conversation really got me thinking. Maybe intentionally slowing down development systems is not such a bad idea…

So what do you think?

This evening I went to the grocery store with my wife, Jill. It was an interesting trip. I have to admit that I took a sneak peek at the shopping list before I agreed to go, and seeing that there were only 7 items on the list, I figured I could earn some extra points without spending too much time. Sneaky, I know, but I am a little weird about optimizing my time. By way of example, one of the main reasons I have a pony tail is because I once calculated how much time it took to get a haircut every month. The calculation went something like this:

Haircut Calculation

Time	Primary Action	Snide Comment
0.50	Commute to the barber shop	Everything is a half hour away in Dallas
0.25	Wait for an open barber chair	Not totally wasted because you could read the 6 year old copies of Field and Stream or the 10 year old copies of Lowrider Magazine – just in case I ever decide to go fishing in my newly restored 57 Chevy
0.75	Actually getting my haircut	Takes a little extra time because of barbers need to give advice on everything from re-wiring a house to colonoscopy
0.25	Paying up	Again takes a little extra time to listen to advice on tattoo removal (not that I have any tattoos, much less any tattoos that need to be removed, but it just seems rude not to act interested, especially to a guy that’s basically in charge of you public image)
0.50	Commute back to wherever	Can take longer if it’s raining because nobody can drive in the rain in Dallas

So that’s a total of 2.75 hours per haircut. Multiply that by 12 times a year and that comes out to 33 hours per year. So that’s almost a whole work week. That just seemed like too much time to commit to such an unrewarding task, so I quit getting my hair cut.

But I digress.

Back to the shopping trip. Like I said, the list had 7 items on it, so I expected a quick trip. Well we get into the store and Jill asks me to get a shopping cart. I’m thinking a little hand basket thingy would be fine, because I’ve seen the list! But I don’t want to tell her that (I’m still being sneaky), so I get the shopping cart. And we proceed down the first isle, stopping every two feet or so to put something in the basket. By the end of the first isle we have at least double the number of items on the list, and by the way, nothing has been marked off the list yet. Just to give you an example, we ended up with 4 bags of chips (Lays Bar-B-Que, Tostitos Scoops, Fritos, and Sun Chips). We’d have had one more (because part of my family likes the Scoop Tostitos and another contingent likes the flat Tostitos) but we had to draw the line somewhere. It wasn’t all my wife’s doing either. We were both throwing stuff into the cart like we were expecting a hurricane to cut off food supplies for the next several weeks. By the time we got out of the store we had 17 bags of groceries. They were the small plastic kind (which I don’t really like – I always say paper when they ask – but I digress again).

I found the whole thing quite amusing, although I don’t think Jill really did.  I asked if I could check the list while we were checking out, just to make sure we hadn’t missed anything. She didn’t really appreciate my humor. (we did forget one of the things on the list by the way)

Anyway, the experience reminded me that trying to get too clever can often backfire. It also reminded me that I am predisposed to make that particular mistake. Implementing a tricky solution to a problem, while it may be intellectually stimulating,  is often not the best approach. The experience reminded of a talk I did a couple of years ago (Creative Problem Solving for Oracle Systems) where I talked about several tendencies that all of us have (to a greater or lesser extent). And how those tendencies can interfere with our ability to solve problems. One of the points of the talk was that being aware of your tendencies can help you avoid pitfalls.

Like Clint Eastwood said, “A Man’s Gotta Know His Limitations”.

One of my favorite holiday treats was my MeeMaw’s fudge brownies. Note: I did a brief poll (only 5 people so not statistically significant) but nevertheless, 100% of the people I surveyed had a grandparent that they called either MeeMaw or PopPa. And 40% had both a MeeMaw and a PopPa. Of course all 5 of the pollees were native Texans. Anyway, here’s what my MeeMaw’s fudge brownies looked like.

Oracle has a long history of baking fudge as well.

So here’s a little Oracle Fudge for you!

11gR1 has 4 parameters with the word fudge in them.

_nested_loop_fudge
_parallelism_cost_fudge_factor
_px_broadcast_fudge_factor
_query_rewrite_fudge

These four “fudge” parameters have been around with the same default values since at least 8.1.7. Maybe the elves will fix these in version 12.

And in keeping with the holiday theme, I’m reminded of the song “My Favorite Things” (often sung at Christmas) that goes:

“blah, blah, blah, blah, blah, blah,
These are a few of my favorite things”
(think Julie Andrews in Sound of Music)

Anyway, here’s a few of my favorite parameters (and my interpretation of what they mean):

db_cache_advice – If you turn this one on, Oracle will tell you what to do with your money.
db_cache_size – And this one will tell you how much money you have.
db_ultra_safe – Oddly enough, this one defaults to OFF. Seems like you’d want your database to be “Ultra Safe”.
ifile – Looks like someone from Apple slipped this one in (you know – iPod, iPhone, iMac, etc…).
large_pool_size – Just how big is your pool?
skip_unusable_indexes – Defaults to TRUE. I guess if you want, you can tell Oracle to use those unusable indexes.
_addm_skiprules – Yeah, rules suck!
_ash_size – Do these pants make my butt look big?
_backup_max_gap_size – How big does the doorway have to be to get your butt through it?
_asm_disk_repair_time – Uh oh, time to repair those disks.
_avoid_prepare – Why get ready ahead of time.
_awr_disabled_flush_tables – Not sure but it sounds stinky.
_awr_sql_child_limit – Population control?
_bloom_pruning_enabled – Trim the roses!
_bwr_for_flushed_pi – Not sure what this one does, but flushing pie seems like such a waste.
_cvw_enable_weak_checking – I’d prefer strong checking please! Get that weak stuff out of here!
_db_aging_cool_count – I used to be cool, I think.
_db_aging_hot_criteria – ???
_db_block_bad_write_check – I hope our database is not writing bad checks!
_db_block_check_for_debug – I think a developer from Chicago named this one (and it should be: _db_block_check_for_the_bug).
_db_cache_crx_check – I don’t know what a crx check is, but cashing any kind of check should be good, right?
_db_large_dirty_queue – Just like in the laundry room at home.
_db_row_overlap_checking – Do your rows overlap? Perhaps we should check that.
_disable_fast_aggregation – Why would anyone use this, “No thanks, I want really slow aggregation”
_dtree_pruning_enabled – Trim D-Tree too while yer at it!
_extended_pruning_enabled – Cut ’em way back!
_disable_recoverable_recovery – Hmmmm???? I guess if you don’t want your recovery to be recoverable you can set this one.
_dummy_instance – I’ve thought this many times (it’s basically the same as the _stupid_database parameter).
_dispatcher_rate_scale – How much are we paying that dispatcher anyway?
_fairness_threshold – My queries should always run faster than everyone else’s, I think that’s fair.
_flashback_fuzzy_barrier – Fuzzy Wuzzy was a bear, Fuzzy Wuzzy had no hair, Fuzzy Wuzzy wasn’t very fuzzy was he.
_gc_defer_time – I’ve wanted to do this many times in the past.
_ges_dd_debug – Sounds like a speech impediment, b,b,but maybe not.
_hard_protection – Well if it was easy, everyone would be doing it.
_imr_avoid_double_voting – They needed this in Florida during the 2004 election.
_in_memory_undo – Same as the _forget parameter.
_kdli_STOP_dba – Keep the DBA from messing up the system.
_kdli_delay_flushes – Don’t flush until a specified threshold is reached.
_kdlwp_flush_threshold – The amount of poo that triggers flushing (see _kdli_delay_flushes).
_kdli_memory_protect – Same as the _dont_forget parameter (i.e. the opposite of the _forget parameter).
_kdli_squeeze – One of my favorite bands.
_kebm_nstrikes – The number of strikes before you are out (defaults to 3 – no joke).
_kebm_suspension_time – How long before convicted felons can return to playing football.
_kill_enqueue_blocker – What the defensive linemen try to do on every play.
_kill_java_threads_on_eoc – I always turn this one on, because anything that kills java threads is OK in my book.
_kfm_disable_set_fence – Good fences make good neighbors.
_kse_signature_limit – The cash advance limit on your credit card.
_kse_snap_ring_size
_ksi_clientlocks_enabled – Can be used on clients when they won’t follow your advice.
_lm_better_ddvictim – Not sure what this one does, but it has the word victim in it, scary!
_lm_master_weight – Set this to get control of your diet.
_lm_tx_delta – Ah the Texas delta, I think it’s some where near Galveston.
_max_exponential_sleep – The older I get, the longer the naps.
_memory_sanity_check – Do I seem crazy to you?
_mv_generalized_oj_refresh_opt – In general, orange juice is refreshing!
_olapi_memory_operation_history_retention – Same as the _dont_forget parameter (see _kdli_memory_protect).
_olap_wrap_errors – When you cut the wrapping paper too short and it won’t go around the present and you have to cut a little strip to cover the gap.
_optimizer_ignore_hints – No matter what you hear in there, no matter how cruelly I beg you, no matter how terribly I may scream, don’t open that door.
_optimizer_random_plan – This one is self evident and has defaulted to TRUE since the CBO first came out.
_optimizer_squ_bottomup – Bottoms Up!
_parallel_fake_class_pct – “The higher you hold your pinky, the fancier you are.” – Patrick from Sponge Bob.
_parallel_syspls_obey_force – The parameter that allows you to get the sysadmins (syspls) to do what you tell them.
_pct_refresh_double_count_prevented – This would have been useful in Florida during the 2004 election.
_pdml_gim_staggered – Gim must have had too much to drink.
_pred_move_around – If your dad was in the air force and you moved from base to base while growing up, setting this parameter will make you feel right at home.
_px_no_stealing – This parameter is set to TRUE by default and it’s actually against the law to change it.
_px_nss_planb – Use this parameter if plan A doesn’t work.
_shrunk_aggs_enabled – I don’t like shrunk aggs, I like the big-uns, wif bacun!
_spin_count – Have you ever seen fans at a baseball game put their heads on the end of a bat and turn circles and then run? You get the idea.
_two_pass_reverse_polish_enabled – I don’t think this one is politically correct.
_use_best_fit – One size fits all does not fit all.
_write_clones – Send a letter to your siblings.
_ultrafast_latch_statistics – Anything that is ultrafast has got to be fantastic!
_xsolapi_densify_cubes – Densify??? I think they made that word up.
_xsolapi_optimize_suppression – Allows us to to keep the common man down as effectively as possible.
_xsolapi_stringify_order_levels – Stringify??? I think they made that word up too!
_xsolapi_use_models – It will make our advertising look better!

Here a few that allow you to turn on (or off) special checks:

_disable_acid_check – My acid is just fine thanks.
_disable_cpu_check – Yep, this machine has got at least one.
_disable_health_check – I don’t need no stinking annual checkup!
_disable_image_check – And I really don’t care how I look!

Here’s a few parameters dealing with death:

_px_execution_services_enabled – Apparently we can set up a special service for executions.
_xsolapi_share_executors – And if you have a lot of killing to do, it’s wasteful to not share executors.
_imr_splitbrain_res_wait – Sounds like it would hurt (if not kill).
_ksv_spawn_control_all – Creates zombies.
_cgs_zombie_member_kill_wait – Specifies how long to wait before slaying zombies!
_imr_evicted_member_kill – Boy I hope I don’t get evicted!
_ksu_diag_kill_time – Killing Time!
_ksuitm_dont_kill_dumper – The rumor is that this one was named by an animal rights activist and it was supposed to be “Don’t Kill Thumper”.
_lm_rcvr_hang_kill – Death by hanging!
_ksv_pool_hang_kill_to – If hanging doesn’t work, drown them in the pool!
_hang_detection – Turn this on so we’ll know when anybody is getting hung.

And just in case the 2000+ parameters in 11g aren’t enough, they have a few spares:

_first_spare_parameter
_second_spare_parameter
_third_spare_parameter
_fourth_spare_parameter
_fifth_spare_parameter
_sixth_spare_parameter
_seventh_spare_parameter

 Anyway, that’s it for now. I hope you have a happy holiday and a …

When I got to work today I walked into my co-worker’s (Michael’s) office and saw this:

Data was scrolling by on the screen in rapid fashion. So I asked him what he was doing and he said he got tired of mashing the inner-butt-n  (that’s the way we say “pressing the the return key” in Texas). Works for me. He could have probably written a custom shell script with proper error checking and whatnot, but why, when the stapler was sitting right there.

I always thought the best programmers were basically lazy. They always seem to find ways to get more done in less time. When I was a young programmer my goal was to write a batch job that would run all month. That way I’d only have to come in on the first to kick it off for the next month. I never quite got there but I had fun trying.

Which reminds me of something that happened at my first programming job. I worked for an oil company that had more money than sense. We had two of everything. We actually had two Cray’s. Anyway, my boss had one of the very first transportable computers, a Grid. The Grids were very futuristic back in 1982. They looked pretty similar to what we have today. So anyway, my boss told me this story after returning from trip with his brand new Grid. He said he was on the airplane and decided to get his new toy out and play with it. So he gets it out of the bag and sets it up on the tray (I guess it was after the flight had taken off due to the electronics restrictions, oh yeah, they didn’t have those then!) – so anyway, he starts getting all the stuff out of the bag and getting organized, and by this time he says everyone within 10 rows is staring at him because no one had ever seen a laptop computer before. And he’s looking around smiling at everyone, thinking yeah this is pretty cool. Then he gets the last part out of the bag, … a power cord.

and he looks at the power cord …

and he looks around the cabin for a place to plug it in …

and he looks at the power cord again …

They didn’t have batteries on those early models. (they didn’t have ethernet jacks either, but they did have a 1200 baud modem BUILT IN!)  So anyway, he sheepishly puts the computer back in the bag and pretends to sleep for the rest of the flight.

Here’s a picture of the Grid computer (notice the wire running out the back <grin>):

My favorite low tech solution though was provided by a friend of mine that got a job right out of college working for an oil company. His first assignment was to fix a bug in an extremely complex reservoir simulation program. Apparently they had been trying to fix the bug for months.  The bug manifested itself by producing a result for one of the calculations that was always off by 1. And they just couldn’t figure out where the error was. They ran test case after test case through it and it was always off by 1. My friend worked on it for a day and then demoed it for them and it worked flawlessly. When asked how he did it, he said “Well, I just went to the end of the program and added 1 to the result”.

Your comments are always welcome.

Here’s a very brief story in response to Doug Burns’ post at his blog calling for stories on Human Error.

My very first job out of college was with a large oil company that had Oracle running on bunch of VAX/VMS systems. We had a lot of code written in FORTRAN with OCI calls (this was back in the early 80’s, so no pre-compilers yet). I was working late one night from home, which in itself was an unusual thing because not too many people were able to work remotely at that time. We only had 1200 baud modems for crying outloud, so it was painful to do much of anything remotely. At any rate, I was working on a program with some kind of iterative processing which took a while to complete. So I’d make a few changes and run it, make a few changes and run it. Well I noticed that the execution time slowed down somewhat and so I went looking to see what else was running on the system. (brief digression: I had become a neophyte sys admin due to my being the Oracle DBA and needing to have some system privileges for doing upgrades and whatnot) So I had a look to see what might be slowing my program down and sure enough there was a batch job running that was really using a lot of cpu. Well I had learned about the ability of VMS to set process priorities and so I thought to myself, “that batch job has all night to run it shouldn’t be slowing me down right now”. So I determined to change the priorities so my program would not be competing so heavily with the batch job. Unfortunately, instead of lowering the priority of the batch process, I jacked the priority of my process way up. (you’ll see why I say “unfortunately” in a minute) So anyway, the priority change worked out great. My program executions began running even faster than they had prior to the batch job kicking off. So I went back to my programming routine. Make a few changes, execute the program, check the results, make a few changes, execute the program, check the results… until I executed the program and it didn’t come back. I remember thinking, “Uh oh, I think I messed up the check for getting out of that loop”. So I thought, “well I’ll just ctl-C out and fix it”. Unfortunately, in a stupendous example of Murphy’s Law,  it was at just this point that my modem lost it’s connection. Great! So I tried to reconnect. The modem was able to establish a connection, but the machine was so busy running the process with the insanely high priority that it didn’t have enough spare CPU to log another process in. (Unlike most systems today, VMS had a very hard priority system. The process with the highest priority basically stayed on the CPU as long as they wanted – oh and by the way, there was only the one CPU) So anyway, the program ended up running most of the night and only stopped because it filled up the disk with a log file that it was writing. Needless to say, the real sys admins were not too happy with me the next morning when I showed up at work.